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Abstract 

S  This  dissertation  describes  an  intelligent,  computer-aided  instructional  (ICAI 
program,  named  GUIDON,  with  capabilities  to  carry  on  a  structured  case  method 
dialogue,  generate  teaching  material  from  production  rules,  construct  and  verify 
a  model  of  what  the  student  knows,  and  explain  expert  reasoning.  The  principle 
objective  of  this  research  has  been  to  convert  MYCIN,  a  knowledge -based  consulta¬ 
tion  program,  into  an  effective  instructional  tool.  GUIDON  combines  the  subject 
matter  knowledge  of  the  consultation  system  with  tutorial  discourse  knowledge, 
while  keeping  the  two  distinct.  - - ^  (continued) 
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-  ’  MYCIN-like  knowledge -based,  consultation  programs  are  designed  to  provide 
expert -level  advice  about  difficult  scientific  and  medical  problems.  High  per¬ 
formance  is  attained  by  interpreting  a  large,  specialized  set  of  facts  and 
domain  relations  that  take  the  form  of  rules  about  what  to  do  in  a  given  cir¬ 
cumstance.  Such  a  rule  base  is  generally  built  by  interviewing  human  experts  to 
formulate  the  knowledge  that  they  use  to  solve  similar  problems  in  their  area 
of  expertise.  While  it  is  generally  believed  that  these  programs  have  significant 
educational  potential,  little  work  has  been  done  to  evaluate  the  problems  of 
realizing  this  potential. 

Using  a  rule  base  for  teaching  provides  a  new  perspective  for  shoving  what 
production  rules  have  to  do  with  human  expertise.  This  dissertation  closely 
examines  the  usefulness  and  adequacy  of  MYCIN's  rules  for  infectious  disease 
diagnosis  as  an  instructional  vehicle:  as  topics  to  be  discussed  in  a  tutorial, 
as  problem-solving  methods  for  understanding  a  student's  behavior,  and  as  skills 
to  be  learned  by  a  student.  It  is  argued  that  MYCIN-like  rule-based  systems 
constitute  a  good  starting  point  for  developing  a  tutorial  program,  but  they  are 
not  sufficient  in  themselves  for  making  knowledge  accessible  to  a  student.  Using! 
GUIDON  as  an  interactive  medium  for  transferring  expertise  provides  a  larger 
context  about  human  cognition;  this  is  reflected  in  our  consideration  of  subject 
matter  representation  and  principles  of  tutorial  discourse. 

The  study  of  subject  matter  representation  focuses  on  knowledge  that  allows 
the  tutor  to  articulate  the  structure,  underlying  principles,  and  strategies  of 
the  domain.  This  dissertation  pays  particular  attention  to  aspects  of  human 
expertise  that  have  not  been  captured  by  the  MYCIN  rule  base,  a  kind  of  investi¬ 
gation  that  has  not  arisen  in  the  construction,  maintenance,  and  use  of  this 
knowledge  base  for  consultation. 

The  study  of  tutorial  discourse  principles  focuses  on  managing  the  dialogue 
to  achieve  economical,  systematic  presentation  of  problem-solving  expertise.  In 
addition,  tutoring  methods  for  opportunistically  presenting  new  material  and  pro 
viding  hints  on  the  basis  of  an  hypothesis  revision  strategy  are  demonstrated. 
GUIDON'S  teaching  and  discourse  expertise  is  represented  as  explicit  rules. 

These  rules  comprise  strategies  for  modeling  the  student,  means  for  sharing 
initiative,  and  knowledge  of  conventional  procedures  for  discussing  a  problem 
in  a  "goal-directed"  way. 

After  the  basic  set  of  tutorial  expertise  was  developed  using  MYCIN's  infec¬ 
tious  disease  rule  set,  some  perspective  on  GUIDON'S  generality  and  domain 
independence  was  attained  by  coupling  it  to  rule  sets  for  other  domains,  including 
an  engineering  application.  Two  experiments  of  this  type  were  performed.  They 
reveal  the  relationship  of  discourse  strategies  to  the  reasoning  structure  of  th^ 
problem  being  discussed. 
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Abstract 


This  dissertation  describes  an  intelligent,  computer-aided  Instructional  (ICAI) 
program,  named  GUIDON,  with  capabilities  to  carry  on  a  structured  case  method 
dialogue,  generate  teaching  material  from  production  rules,  construct  and  verify  a 
model  of  what  the  student  knows,  and  explain  expert  reasoning.  The  principle 
objective  of  this  research  has  been  to  convert  MYCIN,  a  knowledge-based 
consultation  program,  Into  an  effective  instructional  tool.  GUIDON  combines  the 
subject  matter  knowledge  of  the  consultation  system  with  tutorial  discourse 
knowledge,  while  keeping  the  two  distinct. 

MYCIN-llke  knowledge-based  consultation  programs  are  designed  to  provide 
expert-level  advice  about  difficult  scientific  and  medical  problems.  High  performance 
is  attained  by  interpreting  a  large,  specialized  set  of  facts  and  domain  relations  that 
take  the  form  of  rules  about  what  do  to  In  a  given  circumstance.  Such  a  rule  base  is 
generally  built  by  interviewing  human  experts  to  formulate  the  knowledge  that  they 
use  to  solve  similar  problems  in  their  area  of  expertise.  While  it  is  generally  believed 
that  these  programs  have  significant  educational  potential,  little  work  has  been  done 
to  evaluate  the  problems  of  realizing  this  potential. 

Using  a  rule  base  for  teaching  provides  a  new  perspective  for  showing  what 
production  rules  have  to  do  with  human  expertise.  This  dissertation  closely  examines 
the  usefulness  and  adequacy  of  MYCIN's  rules  for  Infectious  disease  diagnosis  as  an 
instructional  vehicle:  as  topics  to  be  discussed  in  a  tutorial,  as  problem-solving 
methods  for  understanding  a  student's  behavior,  and  as  skills  to  be  learned  by  a 
student.  It  is  argued  that  MYCIN-llke  rule-based  systems  constitute  a  good  starting 


Iv 


point  for  developing  a  tutorial  program,  but  they  are  not  aufficient  In  themselves  for 
making  knowledge  accessible  to  a  student.  Using  GUIDON  as  an  interactive  medium  for 
transferring  expertise  provides  a  larger  context  about  human  cognition;  this  is 
reflected  in  our  consideration  of  subject  matter  representation  and  principles  of 
tutorial  discourse. 

The  study  of  subject  matter  representation  focuses  on  knowledge  that  allows 
the  tutor  to  articulate  the  structure,  underlying  principles,  and  strategies  of  the 
domain.  This  dissertation  pays  particular  attention  to  aspects  of  human  expertise  that 
have  not  been  captured  by  the  MYCIN  rule  base,  a  kind  of  Investigation  that  has  not 
arisen  in  the  construction,  maintenance,  and  use  of  this  knowledge  base  for 
consultation. 

The  study  of  tutorial  discourse  principles  focuses  on  managing  the  dialogue  to 
achieve  economical,  systematic  presentation  of  problem-solving  expertise.  In 
addition,-  tutoring  methods  for  opportunistically  presenting  new  material  and  providing 
hints  on  the  basis  of  an  hypothesis  revision  strategy  are  demonstrated.  GUIDON'S 
teaching  and  discourse  expertise  is  represented  as  explicit  rules.  These  rules 
comprise  strategies  for  modeling  the  student,  means  for  sharing  initiative,  and 
knowledge  of  conventional  procedures  for  discussing  a  problem  in  a  "goal-directed" 
way. 

After  the  basic  set  of  tutorial  expertise  was  developed  using  MYCIN's  infectious 
disease  rule  set,  some  perspective  on  GUIDON'S  generality  and  domain  independence 
was  attained  by  coupling  it  to  rule  sets  for  other  domains,  including  an  engineering 
application.  Two  experiments  of  this  type  were  performed.  They  reveal  the 
relationship  of  discourse  strategies  to  the  reasoning  structure  of  the  problem  being 
discussed. 
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INTRODUCTION 


1.1  Task  and  Thesis 

This  dissertation  makes  contributions  to  several  areas  of  research  in  Intelligent 
Computer-Aided  Instruction  (ICAi),  including  means  for  structuring  a  case-method 
dialogue,  generating  teaching  material  from  performance  rules,  constructing  and 
verifying  a  model  of  what  the  student  knows,  and  explaining  expert  reasoning. 

The  ICAI  tutorial  program  described  here,  named  GUIDON,  has  been  built  to 
explore  the  problem  of  converting  a  knowledge-based  consultation  program,  MYCIN, 
into  an  effective  Instructional  tool.  Knowledge-based  consultation  programs 
[Shortliffe,  1976]  [Lenat,  1976]  [Pople,  1976]  are  designed  to  provide  expert- 
level  advice  about  difficult  scientific  and  medical  problems.  High  performance  is 
attained  by  Interpreting  a  large,  specialized  set  of  facts  and  domain  relations 
according  to  the  demands  of  a  particular  problem.  These  facts  and  relations 
constitute  a  knowledge  base  that  is  generally  built  by  interviewing  human  experts  to 
formulate  the  knowledge  that  they  use  to  solve  similar  problems  in  their  area  of 
expertise.  While  It  Is  generally  believed  that  these  programs  have  significant 
educational  potential,  little  work  has  been  done  to  evaluate  the  problems  of  realizing 
this  potential. 

The  GUIDON  system  uses  subject  matter  knowledge  and  tutorial  discourse 
knowledge,  while  keeping  the  two  distinct.  Basic  research  centers  on  two  areas:  (1) 
representation  of  subject  material  to  be  used  In  teaching,  so  that  the  tutor  can 
articulate  the  structure,  underlying  principles  and  strategies  of  the  domain,  (involving 
a  study  of  the  epistemology  of  MYCIN's  knowledge)  and  (2)  management  of  the 
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dialogue  so  as  to  achieve  economical,  systematic  presentation  of  nmblem-soiving 
expertise  (involving  the  creation  of  an  appropriate  and  precise  theory  of  tutoring). 
Some  of  basic  themes  of  this  work,  constituting  the  "thesis"  we  argue,  are: 


I  On  £  x partita 

Current  knowledge-based  programs  like  MYCIN  have  superficial  chains  of 
reasoning  that  capture  little  more  than  the  I/O  behavior  of  human  expertise. 


■  (Corollmryi)  On  Explanation 


Teaching  the  expertise  represented  by  a  knowledge-based  program  requires 
consideration  of  underlying  models  of  the  domain  that  justify  the  rules,  structural 
patterns  that  organize  them,  and  strategies  for  using  them  to  construct  lines  of 
reasoning. 


■  On  Tutoring 


The  natural  language  issues  In  computer-aided  instruction  go  beyond  parsing 
student  input.  There  is  a  body  of  expertise  for  managing  a  dialogue  that  takes  into 
account  conventional  reasoning  patterns,  sharing  Initiative,  and  understanding  the 
student. 


1.2  Features  of  GUIDON 


To  give  some  Idea  about  the  nature  of  the  tutoring  problem  and  what  is  difficult 
about  It,  we  wW  illustrate  the  basic  design  features  of  the  GUIDON  program  with 
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tutorial  excerpts.1  Later  in  this  chapter  we  wiU  consider  assumptions  that  we  have 
made  in  converting  MYCIN  to  a  tutorial  program  and  the  paradigm  for  constructing 
knowledge*based  tutors  that  has  guided  our  research. 


1)  The  program  should  provide  problem-solving  assistance  in  context. 
The  tutor's  guidance  should  be  based  upon  the  student's  partial  solution.  In 
general,  this  is  a  difficult  problem  because  it  requires  that  the  tutor  be 
sensitive  to  the  student’s  current  problem-solving  strategy,  the  kind  of 
advice  he  prefers  (a  hint?  full  details?),  and  be  able  to  articulate  problem¬ 
solving  methods  that  might  be  applied  (a  problem  of  knowledge 
representation). 


In  this  example,  GUIDON  provides  assistance  by  applying  a  solution 
method  (ru!e507)  that  is  relevant  to  the  evidence  discussed  so  far.  In 
this  case  the  selected  method  was  alluded  to  in  an  earlier  hint.  The 
program  has  many  ways  to  present  a  rule  to  the  student,  such  as 
forming  a  question  or  discussing  each  clause  of  the  rule  explicitly.  Here 
GUIDON  demonstrates  the  applicability  of  the  solution  method  by 
showing  how  truth  of  the  single  precondition  that  remains  to  be 
considered  can  be  inferred  from  known  evidence  ( rule$09 ).  The 
inference  is  trivial,  so  it  is  given  directly  rather  than  opened  up  for 
discussion.  GUIDON  then  applies  the  original  method  (rulei07)  and 
comments  about  the  status  of  the  current  subproblem. 


Thus,  providing  assistance  can  involve  applying  a  teaching  strategy  that 
carries  the  solution  of  the  problem  forward.  This  in  turn  requires  being 
able  to  articulate  reasoning  on  the  basis  of  what  the  student  knows, 
according  to  principles  of  economical  presentation. 


1  To  help  the  reader  get  past  the  medical  jargon  to  see  the  Instructional  issues, 
these  excerpts  have  been  cleaned-up,  chiefly  by  substituting  pronouns  for  the 
identifier  "pt538"  and  rephrasing  parameter  descriptions;  the  original  text  is  given  in 
Chapter  3  and  Chapter  9. 
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«  HINT 

Hint:  Look  for  strong,  positive  evidence  that  Diplococcus-pneumoniae 
is  one  of  the  organisms  that  might  be  causing  the  infection. 


**  HELP 

Let's  consider  whether  J.Smith  has  a  head  injury. 

We  already  know  that  the  patient  has  not  had  an  injury  to  the 
central  nervous  system;  this  is  evidence  that  he  does  not  have  a 
head  injury  [RULE509]. 

We  now  have  strong  evidence  that  Diplococcus-pneumoniae  and 
Neisseria-meningitidis  are  organisms  that  might  be  causing  the 
infection  (considering  the  age  of  J.Smith  and  the  fact  that  he  does 
not  have  a  head  Injury  [RULE507]). 

It  remains  for  us  to  consider  other  factors  for  determining  the 
organisms  that  migut  be  causing  the  infection. 


2)  The  program  should  provide  feedback  to  the  student  that  enables  him 
to  evaluate  his  competency.  Here  the  tutor  must  be  able  to  account  for  the 
student's  behavior  in  terms  of  problem-solving  methods  and  strategies  The. 
may  Involve  interpreting  wrong,  but  consistent  behavior  in  terms  of 
misconceptions.  In  correcting  the  student,  the  tutor  may  need  to  refer  back 
to  past  student  actions  and  compare  alternative  approaches.  Providing 
elaboration  in  a  mixed-initiative  exchange  poses  technical  problems  in 
dialogue  management,  such  as  maintaining  context  while  sharing  initiative. 


In  this  excerpt,  the  student’s  hypothesis  for  a  subproblem  is  correct,  but 
after  analyzing  the  previous  history  of  Interaction  with  the  student, 
GUIDON  is  not  sure  that  the  student  really  knows  why  this  hypothesis 
is  correct.  Therefore,  instead  of  accepting  this  answer,  GUIDON 
requests  relevant  factors  to  support  the  hypothesis,  and  probes  further  to 
see  if  the  student  knows  why  an  alternative  hypothesis  is  discredited. 

What  Is  the  type  of  the  Infection? 

**  BACTERIAL 

What  facts  about  this  case  tell  you  that  the  type  of  the  infection 

is  bacterial? 
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(Enter  one  factor  per  line.) 

**  CEREBRAL-FLUID  GLUCOSE  VALUE 
"  PM  NS 

The  percent  of  PMN's  In  the  CBC  Is  not  relevant  for  determining  the 
type  of  the  Infection. 

**  PMNS  IN  THE  CSF 
**  PROTEIN  OF  THE  CSF 


You're  right  about  the  CSF  protein,  the  percent  of  PMNs  in  the  CSF 
and  the  CSF  glucose  value,  however  you  should  have  mentioned  the 
fact  that  the  meningitis  is  partially  treated,  the  WBC  from  the  CSF 
and  the  fact  that  the  simultaneous  blood  glucose  value  is  not  known. 


MYCIN  rejected  some  possibilities  you  did  not  mention: 

What  facts  about  this  case  tell  you  that  the  type  of  the  infection 
is  not  viral? 


> 

3)  The  program  should  make  tutorial  presentations  for  the  purpose  of 
introducing  new  material  to  the  student.  In  a  knowledge-based  tutor, 
presentations  are  generated  solely  from  the  knowledge  base  of  rules  and 
facts.  This  requires  that  the  tutor  have  presentation  methods  that 
opportunistically  adapt  material  to  the  needs  of  the  dialogue.  In  particular, 
the  tutor  has  to  be  sensitive  to  how  a  tutorial  dialogue  fits  together,  what 
kinds  of  interruptions  and  probing  are  reasonable  and  expected  in  this  kind 
of  discourse. 


The  following  example  is  taken  from  a  GUIDON  tutorial  that  uses 
PUFF’s  knowledge  base  for  the  problem  of  pulmonary  function  analysis. 
Here  GUIDON  first  Interrupts  the  student’s  data  collection  to  suggest 
that  he  make  an  hypothesis;  but  the  student  refuses  to  do  so.  The 
program  then  observes  that  there  is  a  particular  problem-solving  method 
that  can  be  applied  that  is  probably  known  to  the  student  (rule020). 
However,  he  probably  can't  apply  the  method  to  this  case  because  he 
doesn’t  know  how  to  verify  a  necessary  precondition.  GUIDON  presents 
the  Inference  that  it  believes  is  unknown  to  the  student  (rule028),  and 
then  asks  him  to  take  this  evidence  forward. 
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'*  TOPIC 

We  are  discussing  the  degree  of  reversibility  of  airway  obstruction 
of  PATIENT-7. 

Hypotheses  you  should  be  considering  are:  none,  insignificant, 
alight,  good  and  significant. 

•*  WHAT  IS  THE  CHANGE  IN  FORCED  EXPIRATORY  VOLUME? 

The  change  in  Forced-Expiratory-Volume  (Isec.)  due  to  dHation  of 
M.Smith  is  31. 

**  WHAT  IS  THE  CHANGE  IN  RESISTANCE? 

The  change  In  Resistance  due  to  dilation  of  M.Smith  is  20. 

We  have  substantially  completed  our  discussion  of  the  degree  of 
reversibility  of  airway  obstruction  of  M.Smith. 

Would  you  like  to  make  a  hypothesis  now  (as  opposed  to  gathering 
more  information)? 

**  NO 


We  will  now  consider  evidence  that  leads  to  a  conclusion  about  the 
degree  of  reversibility  of  airway  obstruction  of  M.Smith. 

Here  is  a  conclusion  that  you  may  not  have  considered.... 

The  fact  that  the  change  In  Forced-Expiratory-Volume  (Isec.)  due  to 
dilation  of  M.Smith  and  the  change  In  Resistance  due  to  dilation  are 
known  Is  evidence  that  there  are  post  bronchodliatlon  test  results 
[RULE028]. 


Given  the  post  bronchodilation  test  results,  does  this  tell  you  that 
the  degree  of  reversibility  of  airway  obstruction  is  good? 

»»  YES 

No,  given  this  fact,  we  can  conclude  that  the  degree  of 
reversibility  of  airway  obstruction  of  M.Smith  Is  significant 
[RULE020]. 

The  conclusion  in  the  question  holds  when  the  change  in 
Forced-Expiratory-Volume  (Isec.)  due  to  dilation  of  the  patient  Is 
between  6  and  10. 


4)  The  program  should  be  respons/ve  to  the  student's  Inltiotlw  in 
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expressing  what  he  knows  and  wants  to  discuss.  In  replying  to  student 
initiative,  a  tutor  needs  to  determine  what  response  makes  sense,  based  on 
what  it  knows  about  the  student's  knowledge  and  shared  goals  for  the 
tutorial  session.  For  example,  when  the  student  takes  the  initiative  by 
saying  he  knows  something,  the  tutor  may  want  to  hold  a  detailed  response 
in  abeyance,  simply  acknowledge  the  student's  remark,  or  probe  him  for  a 
proof.  Selection  among  these  alternative  dialogues  might  require  determining 
what  the  student  could  have  inferred  from  previous  interactions  and  the 
current  situation. 


Here  GUIDON  decides  that  there  is  sufficient  evidence  that  the  student 
knows  the  solution  to  a  relevant  subproblem,  so  detailed  discussion  and 
probing  is  not  necessary. 


•«  I  KNOW  THE  INFECTION  THAT  REQUIRES  THERAPY 

Good.  MYCIN  has  also  made  a  decision. 

That  fact  that  the  cerebral  spinal  fluid  of  J.Smlth  has  been 
cultured  by  attending  physicians  Is  evidence  that  the  Infection  that 
requires  therapy  Is  meningitis  [RULE333]. 

Back  to  our  discussion  of  the  organisms  that  might  be  causing  the 
infection.... 


6)  The  program  should  give  the  student  the  freedom  to  explore  MYCIN's 
reasoning  as  he  desires.  In  laying  out  MYCIN's  reasoning,  the  program  should 
not  act  like  a  passive  information  retrieval  system.  In  addition  to  clearly 
laying  out  data  and  Inferences,  the  tutor  has  to  reason  about  what 
constitutes  reasonable,  expected  elaboration  on  the  basis  of  what  has  been 
previously  discussed. 


In  this  excerpt  GUIDON  provides  details  for  an  inference  (ruleI48)  by 
offering  to  support  necessary  preconditions  that  were  not  considered  in 
the  dialogue  up  to  this  point,  though  they  can  be  inferred  from  known 
data. 

Summary  of  evidence  that  the  type  of  the  Infection  is  bacterial 

(.98)  viral  (-.76)  fungal  (-.83)  tb  (-.83): 

29a.  The  fact  that  the  meningitis  is  partially  treated  (.96)  is 
evidence  that  the  type  of  the  infection  is  bacterial  (.19) 
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[RULE  148] 

29b.  The  WBC  from  the  CSF  and  the  percent  of  PMNs  in  the  CSF  is 
evidence  that  the  type  of  the  infection  is  bacterial  (.9) 
viral  (-.6)  fungal  (-.7)  tb  (-.7)  [RULE601] 


'«  DETAILS  29A 

For  which  of  the  following  factors  would  you  like  to  see  a  review  of 
the  evidence? 

(select  from  the  following): 

1 )  the  infection  which  requires  therapy 

2)  the  fact  that  the  meningitis  Is  partially  treated  (.95) 

itx  2 


Summary  of  evidence  that  the  meningitis  is  partially  treated  (.95): 

32a.  The  infection  which  requires  therapy,  the  fact  that  organisms 
were  not  seen  on  the  stain  of  the  pending  csf  culture  and  the 
time  since  therapy  with  the  eephalothin  was  started  is 
evidence  that  the  meningitis  is  partially  treated  (.96) 

[RULE  146] 

Do  you  want  to  see  RULE  148? 

**  NO 


1 .3  MYCIN:  Why  tutor  at  all? 

MYCIN  Is  a  knowledge-based  program  that  provides  consultations  about 
infectious  disease  diagnosis  and  therapy  [Shortliffe,  1976]  [Davis,  Buchanan,  & 
Shortliffe,  1977].  in  MYCIN,  domain  relations  and  facts  take  the  form  of  rules  about 
what  to  do  in  a  given  circumstance.  A  principle  feature  of  this  formalism  is  the 
separation  of  the  knowledge  base  from  the  Interpreter  for  applying  It.  This  makes  the 
knowledge  accessible  for  multiple  uses,  including  application  to  particular  problems 
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(i.e.,  for  "performance")  and  explanation  of  reasoning  [Davis,  1976].  Converting  the 
MYCIN  system  into  a  tutorial  program  was  suggested  naturally  by  this  capability.  This 
dissertation  closely  examines  the  usefulness  and  adequacy  of  MYCIN's  rules  as  an 
instructional  vehicle:  as  topics  to  be  discussed  in  a  tutorial,  as  problem-solving 
methods  for  understanding  a  student's  behavior,  and  as  skills  to  be  learned  by  a 
student.  It  is  argued  In  this  paper  that  MYCIN-llke  rule-based  expert  systems 
constitute  a  good  basis  for  tutorial  programs,  but  they  are  not  sufficient  in  themselves 
for  making  knowledge  accessible  to  a  student. 

One  can  follow  MYCIN's  reasoning  during  a  consultation  by  using  the  explanation 
system  (by  asking  WHY  case  data  are  being  sought  by  the  program  and  HOW  goals  will 
be  (were)  achieved).  However,  we  believe  that  this  is  an  inefficient  process  for 
learning  the  contents  of  the  knowledge  base.  The  MYCIN  program  Is  only  a  passive 
"teacher":  it  is  necessary  for  the  student  to  ask  an  exhaustive  series  of  questions,  if 
he  is  to  discover  all  of  the  reasoning  paths  considered  by  the  program.  We  believe 
that  most  students  would  not  have  this  persistence,  so  the  wealth  of  expertise  in  the 
knowledge  base  would  be  lost  to  them. 

The  capabilities  of  existing  ICAI  programs  suggested  that  It  would  be  desirable 
to  have  an  active,  intelligent  agent  that  kept  track  of  the  knowledge  flm»  ’>  ■  »* 

presented  to  the  student  In  previous  sessions  and  attempted  to  measure  and  record 
his  competence.  Using  this  individual  record  with  strategies  for  teaching,  the  program 
could  progressively  and  systematically  present  the  knowledge  base  to  the  student 
according  to  his  interests  and  capability  to  advance.  Moreover,  it  turned  out  that  to 
understand  the  student  and  provide  a  progressive  series  of  lessons,  it  would  be 
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necessary  for  the  tutor  to  know  facts  and  relations  that  played  no  part  in  a 
consultation.  "Tutoring"  the  knowledge  base  became  a  focus  for  defining  problems  of 
human  reasoning  and  explanation  that  went  beyond  those  that  were  important  for 
merely  giving  advice. 

1.4  Transfer  of  Expertise 

GUIDON  Is  an  example  of  a  transfer  of  expertise  program.  Previous  work  in 
transfer  of  expertise  that  we  are  building  upon  investigated  the  problem  of 
constructing  and  maintaining  MYCIN's  knowledge  base  through  Interactive  sessions 
with  human  experts  [Oavis,  1976}.  This  is  transfer  of  expertise  from  a  human  to  a 
program.  From  this  perspective,  the  original  consultation  project  investigated  the 
transfer  of  expertise  from  program  to  human  in  the  context  of  giving  advice  (see 
Figure  1.1).  The  goal  of  the  GUIDON  project  is  to  extend  the  transfer  of  expertise 
theme  in  yet  another  direction— from  the  program  to  a  student. 

Observe  that  a  single  knowledge  base  is  coupled  to  different  programs  that 
must  Interact  with  a  human:  this  human  connection  provides  an  important  focus  to  our 
research.  The  question  we  ask  is,  what  should  an  interactive  program  know  if  it  is  to 
effectively  learn  from,  give  advice  to,  or  teach  a  human?  How  should  the  single 
knowledge  base  be  organized  if  it  is  to  be  us«u  for  all  three  of  these  tasks?  While 
this  dissertation  is  concerned  primarily  with  tutorial  interactions,  these  questions 
provide  a  larger  context  about  human  cognition  that  will  be  reflected  In  our 
consideration  of  subject  matter  representation  and  principles  of  tutorial  discourse. 
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USER  STUDENT 


Figure  1.1.  Transfer  of  Expertise:  Learning,  Advising,  and  Teaching 
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GUIDON  Is  not  the  first  program  that  might  be  characterized  as  a  tutor  tor 
transferring  expertise,  indeed,  research  In  individualized,  computer-aided  Instruction 
has  been  greatly  stimulated  in  the  1970s  by  knowledge-based  research.  A  large 
store  of  specialized  facts  and  relations  has  been  used  for  teaching  geography 
[Carbonell,  1970],  meteorology  [Brown,  Burton,  *  Zydbel,  1973]  [Collins,  1976], 
and  electronic  circuit  debugging  [Brown,  Burton,  &  Bell,  1974b][Brown,  Rubinstein, 
&  Burton,  1976].  The  GUIDON  system  represents  the  first  attempt  to  construct  a 
complete  tutorial  program  through  conversion  of  a  knowledge-based  system.  For  this 
reason,  our  research  pays  particular  attention  to  studying  the  aspects  of  human 
expertise  that  have  been  captured  by  the  MYCIN  rule  base,  a  kind  of  investigation 
that  has  not  arisen  In  the  construction,  maintenance,  and  use  of  this  knowledge  base 
for  consultations. 

What  Should  GUIDON  ba  Trying  to  Taach? 

One  of  the  reasons  for  doing  this  work  is  because  we  don’t  tuiiy  understand  the 
problems  of  learning  expertise  like  MYCIN's.  Indeed,  we  don't  know  how  MYCIN's  rules 
correspond  to  what  an  expert  really  knows  or  uses  to  solve  a  problem.  It  is  not  clear 
that  the  organization  and  level  of  abstraction  of  this  performance  knowledge  is 
suitable  for  use  in  a  tutorial  program.  Moreover,  the  rules  may  only  capture  the 
superficial  ,linput/outputn  behavior  of  experts:  what  they  observe,  a  few  Intermediate 
concepts,  and  the  advice  they  give. 

A  naive  view  would  be  that  MYCIN's  rules  are  all  that  a  student  needs  to  know  if 
he  is  to  become  an  expert.  However,  we  can  easily  think  of  other  kinds  of  knowledge 
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that  people  use  all  the  time  when  reasoning  about  a  problem  or  learning  new  material. 
For  example,  there  are  facts  (E.coli  has  the  shape  of  a  rod),  strategies  (consider  the 
accuracy  of  the  data),  models  for  justifying  rules  (some  drugs  Interfere  with 
immunosuppression),  patterns  in  rules  (the  shape  of  an  organism  and  its  staining 
characteristics  tend  to  be  considered  together),  examples  (consider  a  20  year  old 
male  who...),  prototypical  cases,  and  so  on.  in  addition,  we  know  that  human  teachers 
talk  in  terms  of  overviews  or  frameworks  for  structuring  subject  matter.  Courses  are 
frequently  designed  around  notions  of  difficulty,  sophistication,  and  prerequisite 
connections  of  the  subject. 

In  developing  a  tutor  for  MYCIN,  we  will  want  to  consider  these  various  forms  of 
knowledge.  A  basic  question  is:  What  is  It  essential  for  a  tutor  to  be  able  to  articulate 
if  a  problem  solution  is  to  be  understood  and  made  memorable?  Transferring  expertise 
to  a  student  requires  that  we  go  beyond  features  in  the  rule  set  that  were 
incorporated  just  to  make  sure  that  the  consultation  program  got  the  right  answer.  In 
Chapter  6  we  examine  MYCIN's  rules  from  this  perspective. 

The  ICAI  Tutorial  Paradigm 

In  addition  to  the  domain  knowledge  of  the  expert  program,  a  tutorial  program 
requires  teaching  expertise,  such  as  the  ability  to  tailor  the  presentation  of  domain 
knowledge  to  the  student's  competence  and  interests  [Brown,  1S77b].  Even  given 
an  accurate  representation  of  a  human  expert's  knowledge  and  reasoning  procedures, 
we  don't  know  how  to  present  it  to  a  student,  or  what  special  instructional  information 
that  may  play  no  part  in  problem-solving  itself  is  useful. 


T ransftr  of  Expertist 


IS 


In  all  ICAI  programs,  a  student  is  given  some  task  to  solve.  It  may  be  a  game 
requiring  reasoning  skills,  a  diagnostic  problem  (faulty  electronic  circuit  or  patient 
showing  symptoms  of  infection),  or  a  factual  puzzle  to  be  explained  (causes  of  heavy 
rainfall).  Each  of  these  systems  has  an  underlying  "expert"  program  that  can  talk 
about  problems  that  are  posed  to  the  student.  While  the  student  works  on  the  task, 
the  instructional  program  uses  the  expert  program  as  a  relative  measure  of  the 
consistency  and  efficiency  of  the  student's  solution  (his  answer  and  the  overt  steps 
he  took  to  reach  this  answer).  If  the  expert  program  can  articulate  Its  solution  of  the 
task  (talk  about  strategies  for  taking  alternative  steps  to  solve  the  task),  the 
instructional  program  can  provide  guidance  when  the  student  has  difficulty  with  a 
subprobiem  or  does  not  know  what  to  do  next.  One  of  the  major  research  problems  is 
the  design  of  tutorial  strategies:  when  to  say  something  and  what  to  say. 


The  following  three  considerations  seem  central  to  tutorial  discourse: 


1 )  Dialogue  Strategies  -  Problem-solving 


How  is  the  tutor  to  manage  the  discussion  of  a  long  and  difficult 
problem?  What  dialogue  conventions  enable  people  to  guide  a 
discussion  coherently  through  lines  of  reasoning?  When  there  are 
too  many  rules  and  topics  to  discuss  in  the  allotted  time,  how  is  the 
tutor  to  decide  which  to  present  and  which  to  omit?  How  are 
summaries  constructed?  What  sort  of  planning  Is  necessary? 


2)  Diagnostic  Modeling 


How  is  the  tutor  to  determine  what  knowledge  a  student  Is 
using  to  solve  a  problem?  When  expert  reasoning  paths  are  not 
unique  or  a  limited  window  Into  the  student's  thinking  prevents 
monitoring  each  step  of  a  chain  of  reasoning,  how  is  the  tutor  to 
apportion  credit  and  blame  for  the  student's  observable  behavior 
among  the  different  knowledge  sources?  How  Is  a  program  to 
construct  theories  that  account  for  student  misconceptions? 
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3)  Tutoring  Strategies  ■  Correcting  Misconceptions 


What  are  pedagogically  effective  methods  for  bringing  an 
inadequacy  to  the  student's  attention?  What  strategies  can 
usefully  elicit  information  about  a  student's  thinking  process  without 
causing  interference?  How  will  these  strategies  be  related  to  the 
content  of  the  knowledge  base?  How  will  they  be  represented? 


Observe  that  these  questions  arise  in  converting  a  consultation  system 
designed  for  high  performance  into  a  tutorial  program.  The  additional  body  of 
expertise  that  we  must  formulate  for  tutoring  can  be  characterized  as  expertise  for 
transferring  expertise.  Representation  and  development  of  this  expertise  is  a  basic 
focus  for  GUIDON  research. 

1 .6  Design  Assumptions 

The  GUIDON  system  is  designed  to  be  built  on  top  of  any  EMYCIN  knowledge 
base  1  [van  Melle,  1070].  However,  we  have  used  the  original  MYCIN  infectious 
disease  knowledge  base  for  developing  GUIDON.  Besides  making  GUIDON  separate 
from  the  rule  base,  we  have  not  modified  the  rules  In  any  way  during  this  initial 
implementation  of  the  tutorial  system. 

Assumptions  about  the  students  who  will  use  GUIDON  and  the  nature  of  the 
tutorial  dialogue  are  discussed  below. 


1  This  domain-independent  framework  for  building  MYCIN-like  consultation 
systems  Is  described  in  Section  4.2. 


Dtsign  Assumption] 


IT 


The  Student 


In  a  GUIDON  tutorial,  a  student  plays  the  role  of  a  consultant.  The  dialogue  deals 
exclusively  with  a  particular  consultation  that  has  already  been  presented  to  MYCIN. 
Thus,  for  the  infectious  disease  domain,  the  student  is  given  some  Information  about  a 
patient  suspected  to  have  an  infectious  disease,  and  is  expected  to  request  more 
case  data,  as  he  deems  necessary,  to  draw  conclusions  about  the  patient.  The 
purpose  of  a  GUIDON  tutorial  is  to  make  the  student  aware  of  gaps  or  inconsistencies 
in  his  knowledge,  and  to  correct  these  deficiencies. 


The  problems  to  be  solved  by  the  student  require  technical,  specialized  training. 
We  assume  that  the  student  has  some  background  in  the  area  to  be  tutored:  he 
knows  what  problem  is  to  be  solved  (e.g,  to  reach  a  diagnosis  for  a  patient  suspected 
to  have  an  Infectious  disease),  the  real  world  sources  for  observations  to  solve 
problems  (e.g.,  organisms  found  on  cultures  taken  from  a  patient),  and  the  nature  of 
the  problem-solving  methods  (e.g.,  uncertain  judgments  as  opposed  to  numerical 
calculations).  Thus,  he  is  prepared  to  practice  a  basic  set  of  already  acquired  skills. 
GUIDON  will  introduce  "factors"  to  the  student  (representing  a  parametrization  of  the 
real  world  sources)  and  judgmental  rules  for  making  Inferences  that  lead  to  a  solution 
of  the  problem. 


We  assume  that  the  students  will  be  well-motivated  adults  who  are  capable  of  a 
serious,  mixed-initiative  dialogue.  They  will  be  willing  to  follow  the  program's 
guidance,  but  will  want  to  share  In  the  decision  of  what  Is  discussed. 


Furthermore,  we  assume  that  there  are  basic  human  limitations  and  preferences 
that  affect  tutoring  effectiveness.  These  include  considerations  like  memory  and 
learning  capability,  social  conventions  for  carrying  on  a  discussion,  and  Individual 
styles  for  sharing  initiative  and  approaching  new  material. 


Nature  of  the  Dialogue 


We  limit  communication  between  the  program  and  student  to  a  computer  terminal 
that  prints  one  line  at  a  time,  like  a  teletype;  in  particular,  graphical  methods  of 
presenting  information  will  not  be  considered.  The  tutorial  is  expected  to  require 
between  30  and  00  minutes.  In  fact,  the  basic  limitation  of  time  is  an  important 
practical  consideration  that  can  be  expected  to  constrain  the  tutor's  handling  of  the 
dialogue.  We  are  more  concerned  with  the  tutor's  choice  and  relation  of  topics  than 
its  ability  to  parse  and  understand  natural  language  Input. 


The  general  structure  of  the  case  method  dialogue  Is  shown  in  Figure  1 .2. 
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GOAL-DIRECTED  DISCUSSION: 
get  case  data  make  hypotheses 


Figure  1.2.  General  Structure  of  GUIDON  Tutorials 


In  the  current  version  of  GUIDON,  the  student  must  select  a  case  from  the  library 
of  patients  (there  are  over  100  meningitis  cases,  but  as  described  in  Section 
4.4.1,  a  consultation  must  be  run  and  the  results  reconfigured  before  a  tutorial 
can  begin).  Presentation  of  initial  data  involves  describing  the  objects  of  the  case 
(e.g.,  cultures,  organisms)  in  general  terms. 


The  dialogue  from  this  point  is  goal*directed:  Topic  transitions  descend  to  pursue 
deeper  subgoals  and  pop  back,  returning  conclusions.  The  current  subgoal  determines 
the  context  in  which  student  options  are  Interpreted.  For  example,  requests  for  case 
data  that  do  not  pertain  to  this  subgoal  are  considered  to  be  IrrelevH.'i,  and  the 
student  is  told  so.  However,  the  student  can  change  the  goal  under  discussion  at  any 
time  (Section  7.4).  The  topics  of  the  dialogue  are  precisely  the  goals  that  are 
determined  by  applying  MYCIN  rules,  e.g.,  the  type  of  the  Infection. 


During  discussion  of  a  goal,  the  student  repetitively  requests  additional  case 
data,  and  has  a  large  number  of  options  available  for  exploring  MYCIN's  solution  tree. 
He  is  expected  to  state  hypotheses  which  GUIDON  analyzes  in  terms  of  the  case  data 
he  has  received  and  the  conclusions  made  by  MYCIN  from  this  Information. 


1 .0  Method/Guide  to  the  Reader 

Here  we  survey  the  key  methods  for  approaching  the  problems  we  have  outlined. 
Pointers  are  provided  to  relevant  chapters  as  a  guide  to  the  reader. 


Met  hod!  Guide  to  tht  Reader 
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1.8.1  System  Framework 

MYCIN  communicates  its  solution  of  a  problem  as  an  AND/OR  tree  of  goals  and 
rules.  Chapter  4  describes  how  this  tree  is  generated  from  the  output  of  a 
consultation,  and  relates  this  data  structure  to  the  modules  of  the  GUIDON  system. 

1 .6.2  Augmentation  of  Domain  Knowledge 

In  GUIDON  we  have  augmented  the  performance  knowledge  of  rules  by  adding 
three  other  levels:  a  support  level  to  justify  Individual  rules,  a  structural  level  to 
characterize  patterns  in  the  rule  sec,  and  a  strategical  level  to  control  the  application  of 
rules,  in  addition,  the  program  makes  use  of  representational  meta-knowledge  [Davis 
&  Buchanan,  1977]  that  enables  It  to  pick  apart  MYCIN's  rules  and  use  them  to  guide 
the  conversation,  model  the  student’s  understanding,  and  quiz  him.  Chapter  6 
describes  these  levels  of  knowledge  and  their  tutorial  application  In  detail. 

1.6.3  Formalism  of  Transfer  of  Expertise  Expertise 

GUIDON  teaching  expertise  is  represented  as  explicit  rules.  We  view  the  set  of 
tutorial  procedures  as  a  knowledge  base,  and  have  developed  it  in  much  the  same 
way  that  MYCIN's  rule  set  was  constructed.  That  is,  formulation  of  dialogue,  modeling, 
and  tutoring  methods  Is  an  iterative  process  that  requires  frequent  changes  to  the 
program.  For  this  reason,  the  representation  of  tutoring  rules  was  designed  to  make 
them  easy  to  enumerate  and  modify.  The  formalism  of  tutoring  rules  Is  described  in 


Chapter  6. 
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1 .6.4  Discourse  Expertise 

GUIDON  can  be  considered  to  be  a  kind  of  discourse  program.  Discourse 
expertise  includes  knowledge  of  conventional  procedures  for  discussing  a  problem.  In 
GUIDON  these  are  procedures  for  a  goal-directed  case  method  dialogue  (Chapter  6). 
In  addition,  mixed-initiative  tutorial  dialogue  involves  sharing  initiative  with  the  student 
(Chapter  7)  and  constructing  a  model  of  what  he  knows  and  Is  trying  to  do  (Chapter 
8). 

1 .6.6  Experiments  with  Multiple  Knowledge  Bases 

After  the  basic  set  of  tutorial  expertise  had  been  developed  using  the 
Infectious  disease  rule  set,  some  perspective  on  GUIDON'S  generality  was  attained  by 
coupling  it  to  rule  sets  for  other  domains.  Two  experiments  of  this  type  were 
performed;  they  are  described  in  Chapter  0. 


» 


21 


Chapter  2 

Backgrounds  Other  Approaches  to  CAI 


CONTENTS 

2.1  Historical  Overview  of  Educational  Applications  of  Computer 

Technology: 

--  environmental:  free-style  programming 
--  experimental:  playing  with  games  and  sleiulatlons 
"  Instructional:  direct  teaching  (CAI) 

2.1.1  Frame-Oriented  Systems 

2.1.2  Intelligent  Computer-Aided  Instruction 

2.2  Components  of  ICAI  Systems 

2.2.1  The  Expertise  Component 

2.2.2  The  Modeling  Component 

2.2.3  The  Tutoring  Component 

2.3  Medical  CAI 


[Not*:  The  historical  overview  and  description  of  ICAI  system  components  will 
appear  in  A.  Barr  &  E.  A.  Felgenbaum  (Eds.),  The  Handbook  of  Artificial  Intelligence, 
1970.} 


4 


22 


BACKGROUND 


2.1  Historical  Overview  of  Educational  Applications  of  Computer  Technology 

Educational  applications  of  computer  technology  have  been  under  development 
since  the  early  1960s.  These  applications  have  included  scheduling  courses, 
managing  teaching  aids,  and  grading  tests.  The  predominant  application,  however,  has 
involved  using  the  computer  as  a  device  that  Interacts  with  the  student  directly, 
rather  than  serving  as  an  assistant  to  the  human  teacher.  For  this  kind  of  application, 
there  have  been  three  general  approaches. 

The  ad  lib  or  environmental  approach  is  typified  by  Papert’s  LOGO  laboratory 
[Papert,  1970],  that  allowed  students  more  or  less  free-style  use  of  the  machine. 
Students  are  involved  in  programming;  It  is  conjectured  that  learning  problem-solving 
methods  takes  place  as  a  side  effect  of  using  tools  that  are  designed  to  suggest 
good  problem-solving  strategies  to  the  student.  The  second  approach  uses  games  and 
simulations  as  instructional  tools;  once  again  the  student  Is  involved  In  an  activity-for 
example,  doing  simulated  genetic  experiments— for  which  learning  is  an  expected  side 
effect.  The  third  computer  application  in  education  is  computer-assisted  instruction 
(CAI).  Unlike  the  first  two  approaches,  CAI  makes  an  explicit  attempt  to  instigate  and 
control  learning  [Howe,  1973],  This  third  use  of  computer  technology  In  education  is 
the  focus  of  the  following  discussion. 

The  goal  of  CAI  research  Is  to  construct  instructional  programs  that  incorporate 
well-prepared  course  material  In  lessons  that  are  optimized  for  each  student.  Early 
programs  were  electronic  "page-turners"  that  printed  prepared  text  and  simple,  rote 
drills;  and  practice  monitors,  which  printed  problems  and  responded  to  the  student's 
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solutions  using  pre-stored  answers  and  remedial  comments.  In  the  Intelligent  CAI 
(ICAI)  programs  of  the  1970s,  course  material  is  represented  independently  of 
teaching  procedures  so  that  problems  and  remedial  comments  can  be  generated 
differently  for  each  student.  Research  today  focuses  on  the  design  of  programs  that 
can  construct  a  truly  Insightful  model  of  the  student's  strengths,  weaknesses,  and 
preferred  style  of  learning.  It  is  believed  that  Al  techniques  will  make  possible  a  new 
kind  of  learning  environment.  In  this  overview,  we  survey  CAI  techniques  used  by 
past  programs  and  discuss  how  Al  techniques  became  useful  for  ICAI  programs. 

2.1.1  Frame-Oriented  Systems 

The  first  Instructional  programs  took  many  forms,  but  all  adhered  to  essentially 
the  same  pedagogical  philosophy.  The  student  was  usually  given  some  Instructional 
text  (sometimes  without  using  the  computer)  and  asked  a  question  that  required  a 
brief  answer.  After  the  student  responded,  he  was  told  whether  his  answer  was  right 
or  wrong.  The  student's  response  was  sometimes  used  to  determine  his  "path" 
through  the  curriculum,  or  the  sequence  of  problems  he  was  given  (see  [Atkinson  & 
Wilson,  1969]).  When  the  student  made  an  error,  the  program  branched  to  remedial 
material. 

A  courseware  author  attempted  to  anticipate  every  wrong  response  and 
prespecified  branching  to  other  material  based  on  the  underlying  misconceptions  that 
the  author  associated  with  each  wrong  response.  Branching  on  the  basis  of  response 
was  the  first  step  toward  Individualization  of  instruction  [Crowder,  1 962].  This  style 
of  CAI  has  been  dubbed  ad-hoc,  frame-oriented  (AFO)  CAI  by  [Cerbonell,  1970],  to 
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stress  its  dependence  on  author  specified  units  of  information.1  Design  of  ad-hoc 
frames  was  originally  based  on  Skinnerian  stimulus/response  principles.  The  branching 
strategies  of  some  AFO  programs  have  become  quite  involved,  Incorporating  the  best 
learning  theory  that  mathematical  psychology  has  produced  [Atkinson,  1972] 
[Fletcher,  1976]  [Kimball,  1973].  Many  of  these  systems  have  been  used 
successfully  and  are  available  commercially. 

2.1.2  Intelligent  Computer-Aided  Instruction 

In  spite  of  the  widespread  application  of  AFO  CAI  to  many  problem  areas,  many 
researchers  believe  that  most  AFO  courses  do  not  make  the  best  use  of  computer 
technology: 

In  most  CAI  systems  of  the  AFO  type,  the  computer  does  little  more 
than  what  a  programmed  textbook  can  do,  and  one  may  wonder  why 
the  machine  is  used  at  all. ...When  teaching  sequences  are 
extremely  simple,  perhaps  trivial,  one  should  consider  doing  away 
with  the  computer,  and  using  other  devices  or  techniques  more 
related  to  the  task.  ([Carboneil,  1970],  pp.  32;  193) 

In  this  pioneering  paper,  Carboneil  goes  on  to  define  a  second  type  of  CAI  that  is 
known  today  as  knowledge-based  or  intelligent  CAI  (or  ICAI).  Knowledge-based 
systems  and  the  previous  CAI  systems  both  have  representations  of  the  subject 
matter  they  teach,  but  ICAI  systems  also  carry  on  a  natural  language  dialogue  with  the 
student,  and  use  the  student's  mistakes  to  diagnose  his  misunderstandings.  ICAI  has 


1  The  term  frame  as  It  is  used  in  this  context  predates  the  more  recent  usage 
[Minsky,  1976],  and  refers  to  a  block  or  page  or  unit  of  Information  or  text.  Its  use 
here  does  not  refer  to  some  general  form  of  schema. 
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also  been  called  generative  CAI  [Waxier,  1970]  since  It  Is  typified  by  programs  that 
present  problems  by  generating  them  from  a  large  database  representing  the  subject 
material  to  be  taught.  (See  [Koffman  &  Blount,  1973]  for  a  review  of  some  early 
generative  CAI  programs  and  an  example  of  the  possibilities  and  limitations  of  this  kind 
of  program.) 

However,  the  kind  of  program  that  Carbonell  was  describing  In  his  paper  was  to 
be  more  than  Just  a  problem  generator.  Rather,  It  was  to  be  a  computer-tutor  that  had 
the  Inductive  powers  of  its  human  counterparts  and  could  offer  what  Brown  [Brown, 
Rubinstein,  &  Burton,  1976]  calls  a  reactive  learning  environment,  in  which  the  student 
Is  actively  engaged  with  the  Instructional  system,  and  his  interests  and 
misunderstandings  drive  the  tutorial  dialogue.  This  goal  was  expressed  by  other 
researchers  trying  to  write  CAI  programs  that  extend  the  medium  beyond  the  limits  of 
frame  selection: 

Often  it  is  not  sufficient  to  tell  a  student  he  is  wrong  and  indicate 
the  correct  solution  method.  An  intelligent  CAI  system  should  be 
able  to  make  hypotheses  based  on  a  student's  error  history  as  to 
where  the  real  source  of  his  difficulty  lies.  [Koffman  &  Blount, 

1973] 

The  realization  of  the  computer-tutor  has  involved  increasingly  complicated 
computer  programs  and  has  prompted  CAI  researchers  to  use  artificial  Intelligence 
techniques.  Artificial  intelligence  work  in  natural  language  understanding,  the 
representation  of  knowledge,  and  methods  of  Inference,  as  well  as  specific  Al 
applications  like  algebraic  simplification,  calculus  and  theorem  proving,  have  been 


26 


BACKGROUND 


applied  by  various  researchers  toward  making  CAI  programs  that  are  more  intelligent 
and  more  effective.  Early  research  on  ICAI  systems  focused  on  representation  of  the 
subject  matter.  The  benchmark  efforts  include  the  geography  tutor  of  Carbonell  and 
Collins  [Carbonell,  1970]  the  Logic  and  Set  Theory  tutors  by  Suppes  et  al.  [Suppes 
&  Morningstar,  1972]  and  the  electronics  troubleshooting  tutor  of  Brown  and  Burton 
[Brown,  Burton,  &  Bell,  1974b].  The  high  level  of  domain  expertise  in  these 
programs  permits  them  to  be  responsive  in  a  wide  range  of  problem-solving 
interactions. 

These  ICAI  programs  are  quite  different  from  even  the  most  complicated  frame- 
oriented,  branching  program. 

Traditional  approaches  to  this  problem  using  decision  theory  and 
stochastic  models  have  reached  a  dead  end  due  to  their 
oversimplified  representation  of  learning.  ...  It  appears  within 
reach  of  Al  methodology  to  develop  CAI  systems  that  act  more  like 
human  teachers.  [Laubsch,  1976] 

However,  an  Al  system  that  is  expert  in  a  particular  domain  is  not  necessarily  an 
expert  teacher  of  the  material--"ICAI  systems  cannot  be  Al  systems  warmed  over" 
[Brown,  1977b],  A  teacher  needs  to  understand  what  the  student  Is  doing,  not  just 
what  he  is  supposed  to  do.  Al  programs  often  use  very  powerful  problem-solving 
methods  that  do  not  resemble  those  used  by  humans.  In  many  cases,  CAI  researchers 
borrowed  Al  techniques  for  representing  subject  domain  expertise,  but  had  to  modify 
them,  often  making  the  inference  routines  less  powerful  to  force  them  to  follow  human 
reasoning  patterns,  so  as  to  better  explain  their  methods  to  the  student,  as  well  as  to 
understand  his  methods  [Goldberg,  1973]. 
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In  the  mid-1970s,  a  second  phase  In  the  development  of  generative  tutors  has 
been  characterized  by  the  Inclusion  of  expertise  In  the  tutor  regarding  the  student's 
learning  behavior,  as  well  as  tutorial  strategies  [Brown  &  Goldstein,  1977].  Al 
techniques  are  used  to  construct  models  of  the  learner  that  represent  his  knowledge 
in  terms  of  issues  [Burton  &  Brown,  1976])  or  skills  [Barr  &  Atkinson,  1975]  that 
should  be  learned.  This  model  then  controls  tutoring  strategies  for  presenting  the 
material  to  be  learned.  Finally,  some  ICAI  programs  are  now  using  Al  techniques  to 
explicitly  represent  these  tutoring  strategies,  gaining  the  advantages  of  flexibility 
and  modularity  of  representation  and  control  [Brown,  Rubinstein,  &  Burton,  1976] 
[Goldstein.  1977]. 

2.2  Components  for  ICAI  Systems 

The  main  components  of  ICAI  systems  are  problem-solving  expertise  (the 
knowledge  that  the  system  tries  to  impart  to  the  student),  a  student  model  (a 
diagnosis  of  what  the  student  does  and  does  not  know),  and  tutoring  expertise 

(methods  and  strategies  for  presenting  problem-solving  knowledge  to  the  student).1 
Not  all  of  these  components  are  fully  developed  in  every  system.  Because  of  the  size 
and  complexity  of  intelligent  CAI  programs,  most  researchers  tend  to  concentrate  their 
efforts  on  the  development  of  a  single  part  of  what  would  constitute  a  fully  usable 
system.  Each  component  is  described  below. 


1  See  [Self,  1974],  for  an  excellent  discussion  of  the  differences  and 
interrelations  of  the  types  of  knowledge  needed  in  an  intelligent  CAI  program. 


28 


BACKGROUND 


2.2.1  The  Expertise  Component 

The  expert  module  of  an  ICAI  system  generates  problems  and  measures  the 
correctness  of  student  solutions.  Knowledge  of  the  subject  matter  was  originally 
envisioned  as  a  huge  static  database  that  incorporated  all  the  facts  to  be  taught. 
This  idea  was  implicit  In  the  early  drill-and-practice  programs  and  was  made  explicit  in 
generative  CAI.  Representation  of  subject  area  expertise  in  this  way,  using  semantic 
nets,  has  been  useful  for  generating  and  answering  questions  involving  causal  or 
relational  reasoning  [Carboneil  &  Collins,  1973]  [Laubsch,  1975]. 

Recent  systems  have  used  procedural  knowledge  of  the  subject  matter  to  show 
the  student  how  to  do  things  (e.g.,  take  measurements,  make  deductions).  This 
knowledge  is  represented  as  procedural  experts  that  correspond  to  subskills  a 
student  must  learn  in  order  to  acquire  the  complete  skill  [Brown,  Burton,  &  Bell, 
1974b].  Production  rules  have  been  used  to  construct  modular  representations  of 
skills  and  problem-solving  methods  [Goldstein,  1977].  In  addition,  Brown  has  shown 
that  multiple  representations  (e.g.,  a  semantic  net  of  facts  about  an  electronic  circuit 
and  procedures  simulating  the  functional  behavior  of  the  circuit)  are  sometimes  useful 
for  answering  student  questions  and  for  evaluating  partial  solutions  to  a  problem. 
Stevens  and  Collins  [Stevens  *  Collins,  1978]  have  considered  an  evolving  series  of 
metaphorical  "simulation"  models  that  can  be  used  to  reason  about  the  behavior  of 
causal  systems. 


It  should  be  noted  that  not  all  ICAI  systems  can  actually  solve  the  problems  they 
pose  to  a  student.  For  example,  BiP,  the  BASIC  Instructional  Program  [Barr,  Beard,  & 
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Atkinson,  1970]  can't  write  programs,  but  sample  Input/output  pairs  (supplied  by  the 
program  authors)  permit  It  to  test  a  student's  program.  Simliarly,  the  procedural 
experts  in  SOPHIE-1  could  not  debug  an  electronic  circuit.  In  contrast,  the  production 
system  representation  of  subject  knowledge  used  in  WUMPUS  [Goldstein,  1977] 
[Stansfield,  Carr,  A  Goldstein,  1976]  and  GUIDON  enables  the  programs  to  solve 
problems  independently,  as  well  as  to  criticize  student  solutions.  Being  able  to  solve 
problems,  preferably  in  alt  possible  ways,  correctly  and  Incorrectly,  is  important  If  the 
ICAI  program  is  to  make  fine-grained  suggestions  about  the  completion  of  partial 
solutions.  In  this  respect,  the  ability  to  articulate  reasoning  [Goldstein  A  Papert, 
1977]  in  an  understandable  way  is  a  useful  dimension  for  comparing  ICAI  systems 
(Section  10.2.2). 

All  ICAI  systems  are  distinguished  from  earlier  approaches  by  the  separation  of 
teaching  strategies  from  the  subject  expertise  to  be  taught.  However,  the  separation 
of  subject-area  knowledge  from  instructional  planning  requires  a  structure  for 
organizing  the  expertise  that  captures  the  difficulty  of  various  problems  and  the 
Interrelationships  of  course  material.  Modeling  a  student's  understanding  of  a  subject 
is  closely  related  conceptually  to  figuring  out  a  representation  for  the  subject  itself  or 
for  the  language  used  to  discuss  it. 

Trees  [Koffman  A  Blount,  1973]  and  lattices  showing  prerequisite  interactions 
have  been  used  to  organize  the  introduction  of  new  knowledge  or  topics.  In  BIP  this 
lattice  took  the  form  of  a  curriculum  net  that  related  the  skills  to  be  taught  to  example 
programming  tasks  that  exercised  each  skill.  Goldstein  called  the  lattice  a  syllabus  in 
the  WUMPUS  program  and  emphasized  the  developmental  path  that  a  learner  takes  in 
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acquiring  new  skills.  For  arithmetic  skills  used  In  WEST,  Burton  and  Brown  use  levels 
of  issues.  Issues  proceed  from  the  use  of  arithmetic  operators  to  strategies  for 
winning  the  game,  to  meta-level  considerations  for  improving  performance.  Burton  and 
Brown  believe  that  when  the  skills  are  "structurally  Independent,"  the  order  of  their 
presentation  is  not  particularly  crucial.  This  representation  is  useful  for  modeling  the 
student's  knowledge  and  coaching  him  on  different  levels  of  abstraction.  Stevens  and 
Collins  have  argued  further  that  a  good  human  tutor  does  not  merely  traverse  a 
predetermined  network  of  knowledge  in  selecting  material  to  present.  Rather,  it  is  the 
process  of  ferreting  out  student  misconceptions  that  drives  the  dialogue. 

2.2.2  The  Modeling  Component 

The  modeling  module  is  used  to  represent  the  student's  understanding  of  the 
material  to  be  taught.  Much  recent  ICAI  research  has  focused  on  this  component.  The 
purpose  of  modeling  the  student  Is  to  make  hypotheses  about  his  misconceptions  and 
nonoptimal  strategies  so  that  the  tutor  module  can  point  them  out  and  suggest  why 
they  are  wrong.  It  is  advantageous  for  the  system  to  be  able  to  recognize  alternate 
ways  of  solving  problems,  including  the  Incorrect  methods  that  the  student  might  use 
based  on  systematic  misconceptions  about  the  problem  or  on  Inefficient  strategies. 

Some  of  the  original  frame-oriented  systems  used  mathematical  stochastic 
models  of  the  student,  but  this  approach  failed  because  It  only  modeled  the  probability 
that  a  student  would  give  a  specific  response  to  a  stimulus.  In  general,  knowing  the 
probability  of  a  response  is  not  the  same  as  knowing  what  a  student  is  thinking  about; 
It  has  little  diagnostic  power  [(.aubsch,  1976]. 
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Typical  use  of  AI  techniques  for  modeling  student  knowledge  Includes  simple 
pattern  recognition  applied  to  the  student's  response  history,  and  flags  in  the  subject 
matter  semantic  net  or  rule  base  representing  areas  that  the  student  has  mastered. 
In  these  ICAI  systems,  a  student  model  is  formed  by  comparing  the  student's  behavior 
to  that  of  the  computer-based  "expert"  in  the  same  environment.  The  modeling 
component  marks  each  skill  according  to  whether  evidence  indicates  that  the  student 
knows  the  material  or  not.  Goldstein  has  termed  this  component  an  overlay  model 
because  the  student's  understanding  is  represented  completely  in  terms  of  the 
expertise  component  of  the  program. 

In  contrast,  another  approach  is  to  model  the  student's  knowledge  not  as  a 
subset  of  the  expert's,  but  rather  as  a  perturbation  or  deviation  from  the  expert's 
knowledge,  that  is,  In  terms  of  bugs.  There  is  a  major  difference  between  the  overlay 
and  "buggy"  approaches  to  modeling:  In  the  latter  approach  It  Is  not  assumed  that  the 
student  reasons  as  the  expert  does,  but  simply  knows  less.  Thus,  the  student's 
reasoning  can  be  substantially  different  from  expert  reasoning.  How  to  represent, 
diagnose,  and  generate  these  deviations  is  a  major  problem. 

Other  elements  that  might  be  included  In  the  student  model  are  preferred  modes 
for  Interacting  with  the  program,  a  rough  characterization  of  the  student’s  level  of 
ability,  a  consideration  of  what  he  seems  to  forget  over  time,  and  an  Indication  of 
what  his  goals  and  plans  seem  to  be  for  learning  the  subject  matter. 

Major  sources  of  evidence  used  to  maintain  the  student  model  can  be 
characterized  as:  (a)  implicit,  from  student  problem-solving  behavior;  (b)  explicit, 
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from  direct  questioning  of  the  student;  (c)  historical,  from  assumptions  based  on  the 
■  student's  experience;  and  (d)  structural,  from  assumptions  based  on  some  measure 
of  the  difficulty  of  the  subject  material  [Goldstein,  1970].  Historical  evidence  is 
usually  determined  by  asking  the  student  to  rate  his  level  of  expertise  on  a  scale 
from  "beginner"  to  "expert".  Early  programs  like  SCHOLAR  used  only  explicit 
evidence.  Recent  programs  have  concentrated  on  inferring  implicit  evidence  from  the 
student's  problem-solving  behavior.  This  approach  is  complicated  because  it  is  limited 
by  the  program's  ability  to  recognize  and  describe  the  strategies  being  used  by  the 
student.  Specifically,  when  the  expert  program  Indicates  that  an  inference  chain  is 
required  for  a  correct  result,  and  the  student's  observable  behavior  is  wrong,  how  is 
the  modeling  program  to  know  which  of  the  Intermediate  steps  arq  unknown  or 
incorrectly  applied  by  the  student?  This  is  the  apporlionmtnt  of  credit  Want  problem ;  it 
has  been  an  important  focus  of  WEST  research. 

Because  of  inherent  limitations  in  the  modeling  process,  it  is  useful  for  a  critic  in 
the  modeling  component  to  measure  how  closely  the  student  model  describes  the 
student's  behavior.  Extreme  inconsistency  or  an  unexpected  demonstration  of 
expertise  in  solving  problems  might  indicate  that  the  representation  being  used  by  the 
program  does  not  capture  the  student's  approach.  Finally,  Goldstein  has  suggested 
that  the  modeling  process  should  attempt  to  measure  whether  or  not  the  student  is 
actually  learning  and  to  discern  what  teaching  methods  are  most  effective.  Much 


work  remains  to  be  done  in  this  area. 
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2.2.3  The  Tutoring  Componant 

The  tutoring  module  of  ICAI  systems  must  integrate  knowledge  about  natural 
language  dialogues,  teaching  methods,  and  the  subject  area  to  be  taught.  This  is  the 
module  that  communicates  with  the  student:  selecting  problems  for  him  to  solve, 
monitoring  and  criticizing  his  performance,  providing  assistance  upon  request,  and 
selecting  remedial  material.  This  module  must  deal  with  questions  like  "When  is  it 
appropriate  to  offer  a  hint?”  or  "How  far  should  the  student  be  allowed  to  go  down 
the  wrong  track?" 

These  are  just  some  of  the  problems  which  stem  from  the  basic  fact 
that  teaching  is  a  skill  which  requires  knowledge  additional  to  the 
knowledge  comprising  mastery  of  the  subject  domain.  [Brown, 

1977b] 

This  additional  knowledge,  beyond  the  representation  of  the  subject  domain  and  the 
student  knowledge,  Is  about  how  to  teach. 

Most  ICAI  research  has  explored  teaching  methods  based  on  diagnostic  modeling 
in  which  the  program  debugs  the  student's  understanding  by  posing  tasks  and 
evaluating  his  response  [Collins,  1976]  [Brown,  et  ai.,  1975]  [Koffman  &  Blount, 
1 973].  The  student  is  expected  to  learn  from  the  program's  feedback  of  which  skills 
he  uses  wrongly,  which  he  does  not  use  (but  could  use  to  good  advantage),  etc. 
Recently,  there  has  been  more  concern  with  the  possibility  of  saying  just  the  right 
thing  to  the  student  so  that  he  will  realize  his  own  inadequacy  and  switch  to  a  better 
method  [Carr  <■  Goldstein,  1977]  [Burton,  1979]  [Norman,  Gentnar,  &  Stevens, 
1976].  This  new  direction  is  based  on  attempts  to  make  a  bug  "constructive"  by 
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establishing  for  the  student  that  there  is  something  inadequate  in  his  approach,  and 
giving  enough  information  so  that  the  student  can  use  what  he  already  knows  to  focus 
on  the  bug  and  characterize  it  so  that  he  avoids  this  failing  In  the  future. 

However,  it  is  by  no  means  clear  how  "just  the  right  thing"  is  to  be  said  to  the 
student.  We  do  know  that  it  depends  on  having  a  very  good  model  of  his 
understanding  process  (the  methods  and  strategies  he  used  to  construct  a  solution). 
Current  research  is  focusing  on  means  for  representing  and  isolating  the  bugs 
themselves  [Stevens,  Collins,  It  Goldin,  1978]  [Brown  &  Burton,  1978]. 

Another  approach  Is  to  provide  an  environment  that  encourages  the  student  to 
think  In  terms  of  debugging  his  own  knowledge.  In  one  BIP  experiment  [Wescourt,  et 
el.,  1978],  explicit  debugging  strategies  (for  computer  programming)  were  conveyed 
in  a  written  document  and  then  a  controlled  experiment  was  undertaken  to  see 
whether  this  fostered  a  more  rational  approach  for  detecting  faulty  use  of 
(programming)  skills. 

Brown,  Collins  and  Harris  [Brown,  1977b]  suggest  that  one  might  foster  the 
ability  to  construct  hypotheses  and  test  them  (the  basis  of  understanding  in  their 
model)  by  setting  up  problems  in  which  the  student's  first  guess  is  likely  to  be  wrong, 
thus  "requiring  him  to  focus  on  how  he  detects  that  his  guess  Is  wrong  and  how  he 
then  intelligently  goes  about  revising  it.” 

The  Socratic  method  used  in  WHY  [Collins,  1976]  involves  questioning  the 
student  In  a  way  that  will  encourage  him  to  reason  about  what  he  knows,  and  so 
modify  his  conceptions.  The  tutor's  strategies  are  constructed  by  analyzing  protocols 
of  real-world  student/teacher  interactions. 
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Another  teaching  strategy  that  has  been  successfully  Implemented  on  several 
systems  is  called  coaching  [Goldstein,  1977].  Coaching  programs  are  not  designed  to 
cover  a  predetermined  lesson  plan  within  a  fixed  time  (in  contrast  with  SCHOLAR). 
Rather,  the  goal  of  coaching  is  to  develop  the  acquisition  of  skill  and  general  problem 
solving  abilities  by  by  engaging  the  student  in  a  game  and  unobtrusively  making 
suggestions  for  improving  his  play.  In  a  coaching  situation,  the  immediate  aim  of  the 
student  is  to  have  fun,  and  skill  acquisition  is  an  indirect  consequence.  WUMPUS  and 
WEST  are  both  coaching  programs. 

Socratic  tutoring  and  coaching  represent  different  styles  for  communicating  with 
the  student.  All  mixed*initiative  tutoring  involves  following  some  dialogue  strategy. 
This  will  include  decisions  about  when  and  how  often  to  question  the  student,  and 
methods  for  presentation  of  new  material  and  review.  For  example,  by  design,  a 
coaching  program  Is  not  intrusive,  and  only  rarely  lectures.  On  the  other  hand,  a 
Socratic  tutor  questions  repetitively,  requiring  the  student  to  pursue  certain  lines  of 
reasoning.  Recent  ICAI  research  has  turned  to  making  explicit  these  alternative 
dialogue  principles.  Collins  has  pioneered  the  careful  articulation  and  Investigation  of 
teaching  strategies,  using  production  rules  as  a  stylistic  convention  for  describing  the 
strategies  used  by  his  program. 

In  general,  ICAI  programs  have  only  begun  to  deal  with  the  problems  of 
representing  and  acquiring  teaching  expertise  and  of  determining  how  this  knowledge 
should  be  integrated  with  general  principles  of  discourse. 
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2.3  Medical  CAI 

The  literature  for  medical  CAI  systems  Is  extensive.  [Hoffer,  et  al.,  1976] 
provides  a  good  overview.  Not  ail  of  the  programs  reported  can  be  characterized  as 
frame-oriented;  some  programs  use  probability  tables  to  generate  "cases"  (a  patient 
with  a  specific  problem),  and  use  differential  diagnosis  to  analyze  the  student's 
response  and  provide  assistance  [Entwisle  &  Entwisle,  1963][Steele,  et  al.,  1978], 
Most  researchers  address  the  following  set  of  issues:  realism  of  the  case,  nature  of 
the  dialogue,  feedback  and  pedagogy,  and  the  problem  of  assembling  a  variety  of 
cases.  The  successful  use  of  the  case  method  in  these  programs  [Harless,  et  al„ 
1971]  [Swets  &  Feurzeig,  1966]  [Webar  &  Hagamen,  1972]  has  most  directly 
influenced  the  design  of  GUIDON. 
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This  chapter  Includes  the  entire  typescript  of  a  GUIDON  tutorial  session.  The 
tutorial  shown  here  is  used  to  illustrate  the  program's  use  of  domain  knowledge, 
tutoring  rules,  student  initiative,  and  modeling  discussed  in  later  chapters.  These 
issues  are  brought  out  in  the  typescript  by  annotations  signified  by  braces  ({});  the 
student's  responses  to  GUIDON  are  In  BOLDFACE  and  follow  the  double  asterisks  (**). 
Later  references  to  this  typescript  will  usually  mention  "the  protocol"  and  a  page 
number.  The  item  of  interest  on  a  referenced  page  Is  indicated  by  the  pointing  hand 
ea).  a  good  way  to  become  familiar  with  the  GUIDON'S  features  Is  to  scan  the 
protocol,  reading  these  comments.  Other  sections  of  this  chapter  describe  how  the 
typescript  was  produced  and  a  summary  of  the  student's  (verbal)  comments  during 
the  tutorial. 

3.1  The  Experimental  Setting 

The  purpose  of  the  experiment  was  to  produce  a  typescript  that  Illustrated  most 
of  the  program's  capabilities,  specifically  for  inclusion  in  this  dissertation.  It  was 
deemed  important  that  the  student  have  an  adequate  background  to  actually  work  out 
a  case  posed  by  GUIDON.  This  allows  us  to  directly  observe  how  the  student's 
approach  meshes  with  the  program's  initiative  and  available  options. 

The  student  who  used  GUIDON  in  this  tutorial  was  a  second-year  medical 
student.  He  had  no  previous  experience  with  this  program  or  MYCIN's  rules,  though  he 
had  an  undergraduate  background  in  computer  science.  Consequently,  the  student 
was  familiar  with  the  terminal  and  the  mechanics  of  interacting  with  a  computer 
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program,  so  he  was  able  to  focus  directly  on  the  problemsolving  task  itself.  The 
author  monitored  his  use  of  the  program,  sometimes  helping  him  to  translate  his 
questions  into  program  options.  A  large  cardboard  diagram  that  summarized  the  30 
options  available  to  the  student  was  always  on  display.  In  addition,  the  student  made 
frequent  reference  to  the  GUIDON  handbook  for  MYCIN  (Appendix  C)  which  he  read  the 
night  before. 

The  session  took  nearly  4  hours  because  the  student  was  encouraged  to 
verbalize  his  thoughts  throughout.  The  author  and  student  also  spent  some  time 
discussing  pros  and  cons  of  the  program's  remarks  (these  are  summarized  in  Section 
3.4).  The  reader  will  observe  that  the  student  first  exhibits  uncertainty  about 
what  to  do,  shifts  to  an  aggressive  attack  on  the  problem,  and  finally  runs  out  of 
leads,  too  tired  to  explore  MYCIN's  solution  in  much  detail. 

The  typescript  shown  here  was  produced  by  correcting  several  program  bugs  1 
and  then  rerunning  the  tutorial,  repeating  the  student's  original  input  exactly.  There 
is  one  exception:  since  the  student  answered  all  of  the  program's  quizzes  correctly, 
one  answer  was  modified  to  prove  that  GUIDON  could  actually  deal  with  an  Incorrect 
response  (page  62). 

Finally,  the  phrase  "organisms  (other  than  those  seen  on  cultures  or  smears) 
which  might  be  causing  the  infection"  has  been  replaced  by  "organisms  that  might  be 


1  For  example,  a  new  constraint  was  added  to  the  t-rules  for  selecting  quiz 
types  to  prevent  a  nonsensical  question,  the  control-0  option  for  aborting  output  was 
fixed,  and  other  similar,  local  changes  were  made  that  did  not  change  the  basic  flow 
of  the  dialogue.  No  t-rules  for  modeling  the  student  or  guiding  the  dialogue  were 
modified. 
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causing  the  infection.”  This  is  one  of  MYCIN's  fixed-text  parameter  descriptions  that 
a  tutorial  system  may  need  to  word  In  different  ways  in  different  contexts. 

3.2  Program  Features  to  Look  For 

The  current  version  of  GUIDON  can  be  characterized  as  an  unobtrusive  reasoning 
assistant.  Through  a  wide  variety  of  simple  options,  the  program  provides  easy 
access  to  a  large  knowledge  base,  helping  the  student  to  measure  the  adequacy  of 
his  understanding  of  the  problem.  For  example,  observe  how  the  "data  retrieval" 
options  USE,  CONCLUDE,  FACTORS,  PENDING  (protocol  page  74,  page  68,  page  65. 
page  72)  and  statements  of  some  of  MYCIN's  conclusions  (preceded  by  ">>>") 
indirectly  challenge  the  student  to  address  problem  subtasks. 

The  student  usually  controls  the  solution  of  the  problem:  he  determines  when  a 
topic  is  changed  and  the  order  in  which  case  data  are  considered.  Specifically,  the 
DISCUSS  option  changes  the  topic  with  later  return  to  the  current  topic.  The  TELLME 
option  effectively  terminates  discussion  of  a  topic.  These  features  demonstrate  how 
GUIDON  provides  for  and  copes  with  student  initiative,  the  subject  of  Chapter  7. 

GUIDON  takes  the  initiative  by  offering  orientation  to  new  topics  at  several 
times.  It  presents: 

1 )  the  "goal  rule"  of  the  task  (see  Appendix  D) 

2)  a  sketch  of  the  entire  subgoal  tree  (Appendix  D) 

3)  a  "model"  of  the  typical  factors  to  consider  for  a  "major" 
goal  (page  47) 
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4)  FACTORS  to  consider,  the  first  time  help  is  requested  for  a 
goat  (page  49) 

6)  introduction  of  a  rule  in  terms  of  the  evidence  it  will 
contribute  to  what  is  already  known  (page  68). 

Tutorial  rules  explicitly  control  when  this  information  Is  offered.  For  example,  default 

heuristics  now  limit  presentation  of  the  goal  rule  to  a  new  user  of  the  program.  In 

addition,  following  a  method  we  call  opportunistic  tutoring,  GUIDON  interrupts  the 

student  to  question  him  about  what  he  is  doing  (page  67)  and  to  present  Information 

that  broadens  his  knowledge  beyond  the  needs  of  the  case  he  is  solving  (page  77). 

Tutoring  rules  and  principles  for  guiding  the  dialogue  in  this  way  are  considered  In 

Section  6  2.3.3. 

Finally,  the  program  maintains  a  model  of  what  the  student  knows,  as  well  as 
some  limited  Information  about  topics  In  which  he  Is  interested.  Presentation  methods 
are  tuned  to  this  model,  as  evidenced  by  a  statement  of  a  result  rather  than  a  quiz 
about  it  (page  76)  and  response  to  a  student  hypothesis  (page  56)  that  reflects 
the  tutor's  measure  of  the  student's  understanding.  The  contents,  maintenance  and 
use  of  the  student  model  are  considered  In  Chapter  8. 
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3.3  The  Protocol 


March  25,  1979  12:57PM  in  <CLANCEY>GUIDON.SAV;90324  by  CLANCEY 
This  is  the  tutor  for  MYCIN. 

Instructions  (Y  or  N)? 

»«  YES 

Brief  Description  of  the  Tutorial  Session 

You  are  to  play  the  role  of  consultant  for  a  patient  suspected  to 
have  an  infection.  After  Initial  data  has  been  presented,  you  should  request 
more  information  about  the  patient,  decide  which  organisms  (if  any)  require 
therapy,  and  prescribe  treatment.  The  tutor  will  direct  you,  according  to 
the  profile  you  set.  Its  behavior  can  vary  from  leaving  you  alone  until  you 
are  ready  to  make  a  diagnosis,  to  directing  your  thinking  at  every  step. 

Feel  free  to  change  your  profile  or  request  HELP  to  solve  the  problem  at  any 
time.  Use  a  question  mark  (?)  whenever  you  want  to  see  the  list  of  options 
open  to  you.  Remember  that  the  purpose  of  this  session  is  to  present  MYCIN' 
rules  of  infectious  disease  diagnosis  and,  in  the  second  part,  the  criteria 
of  therapy  selection. 

Type  control-G  to  call  GUIDON.  You  will  receive  the  initiative  after 
the  next  question  Is  asked. 


Please  enter  your  initials: 

««  STU 


Well,  STU,  it  looks  like  we  don't  have  a  file  on  you., 
Please  enter  your  name  and  address: 

(terminate  with  control-Z) 

*«tz 
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*T]Sess/on  Profit m 


{The  profile  is  used  to  initialize  the  student  model  for  a  new  student  and  control  the 
tutor’s  remarks  about  MYCIN's  reasoning.  Options  include  printing  nothing,  a  hint 
(or  general  statement  about  a  result),  or  the  actual  conclusion  made  by  the  program.) 


The  following  questions  concern  the  SESSION  PROFILE. 

Do  you  want  Instructions  for  using  the  profile? 

**  NO 

Standard  setting?  (type  ?  for  help) 

**  no 

On  a  scale  from  0  to  4,  how  do  you  rate  your  problem-solving  experience 
in  this  field  [0=naive  (no  training);  1®some  experience  (beginning 
student);  2sconsiderable  experience  (advanced  student);  3spractitioner; 
4=specialist]? 

Suggested  setting:  t 

**  J 

Should  MYCIN  tell  you  when  a  rule  succeeds? 

Suggested  setting:  NO 

**  NO 

Should  MYCIN  tell  you  when  a  rule  fails? 

Suggested  setting:  NO 

**  NO 

Which  rule  conclusions  should  MYCIN  mention  during  the  session? 
Suggested  setting:  HINTONLY 
.  **  HINTONLY 

What  should  MYCIN  tell  you  when  a  final  conclusion  for  a  factor  is 
known? 

Suggested  setting:  HINTONLY 

**  HINTONLY 

Should  the  default  lesson  plan  be  used? 

Suggested  setting:  YES 

”  YES 

Should  DEBUGGING  information  be  printed? 

Suggested  setting:  NO 
-NO 
okay 


1 


1 


1 
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{Next,  files  containing  case  data,  an  ANDIOR  tree  representation  of  MYCIN’s 
solution,  and  records  from  the  consultation  are  loaded.  After  a  few  minutes  wait,  a 
basic  outline  of  the  case  is  given  to  the  student.  This  is  done  using  MYCIN's  list  of 
" main  properties''— identifying  data  it  always  collects  about  an  object,  eg.,  the  site  of 
a  culture,  the  name,  age,  and  sex  of  a  patient.  In  this  case  there  is  one  culture  that 
has  been  sent  to  the  labs,  and  one  drug  that  the  patient  is  still  receiving;  an  infection 
is  suspected.  The  format  for  displaying  initial  data  is  simple  in  this  first  version  of 
GUIDON  Later  development  will  need  to  take  into  account  methods  for  more  concise 
presentation,  perhaps  according  to  domain-specific  convention i.} 


Tutorial  Session  25-Mar-79  12:59:18 

Which  case? 

**  538 

Loading  <CLANCEY>PATIENT.538:5... 
Loading  TUTORIAL.538... 

File  created  6-Feb-79  18:49:34 
Loading  DIAGNOSIS.638... 

File  created  6-Feb-79  18:60:01 

[consultation  of  30-Jan-77  10:16PM] 

Initial  Case  Oata 


PATIENT-638: 

The  name  of  PATIENT-638  is  J.Smith. 

The  age  of  J.Smith  Is  34  years. 

The  sex  of  J.Smith  is  male. 

The  race  of  J.Smith  Is  Caucasian. 

There  was  1  pending  cultures  of  J.Smith,  as  follows... 
CULTURE- 1: 

The  site  of  CULTURE- 1  is  csf. 

The  time  since  the  specimen  for  the  pending  csf  culture 
was  obtained  is  1 0.2  hours. 

A  smear  of  the  pending  csf  culture  was  examined. 

There  was  1  current  drugs  of  J.Smith,  as  follows... 

DRUG-1: 

The  name  of  DRUG-1  is  cephalothin. 

The  route  of  administration  for  the  cephalothin  is  Iv. 

The  time  since  therapy  with  the  cephalothin  was  started 
Is  1 .06  weeks. 

There  was  1  suspected  Infections  of  J.Smith,  as  follows... 
INFECTION- 1 :  [No  initial  data  will  be  presented] 
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{Using  the  ANDIOR  trtt  and  initial  case  data,  GUIDON  fires  domain  rules  in  a 
forward  direction.  Directed  by  the  session  profile,  GUIDON  mentions  some  of 
MYCIN’s  conclusions  (2a-4a)  (’>»’  indicates  such  a  remark  throughout  this 
protocol).  However,  note  that  simplification  rules  have  reduced  the  number  of 
statements,  such  as  omitting  comments  about  intermediate  conclusions  when  there  is 
only  one  rule  that  succeeds.  Comment  4a  is  an  artifact  of  MYCIN’s  representation- 
"organisms  of  the  patient *  is  the  set  union  of  organisms  associated  with  each 
culture.) 


[Now  giving  initial  case  data  to  MYCIN...  this  may  take  a  few  minutes.] 


»>  2a.  MYCIN  has  made  a  FINAL  CONCLUSION  about  organisms  noted  on  smears  of 
the  pending  csf  culture. 

>>>  3a.  MYCIN  has  made  a  FINAL  CONCLUSION  about  whether  organisms  were  seen 
on  the  stain  of  the  pending  csf  culture. 

»>  4a.  MYCIN  has  made  a  FINAL  CONCLUSION  about  organisms  noted  on  smears  of 
J.  Smith. 


[MYCIN  is  now  considering  additional  questions  to  ask....] 


{ Each  parameter  in  the  ANDIOR  tree  Is  now  scanned  to  determine  if  it  is  now 
known,  or  whether  case  data  will  need  to  be  requested.  5 a  and  6a  are  known  because 
they  are  not  relevant  to  a  case  that  doesn’t  have  positive  cultures.) 

»>  6a.  MYCIN  has  made  a  FINAL  CONCLUSION  about  the  associated  infections 
for  which  therapy  should  cover. 

»>  6a.  MYCIN  has  made  a  FINAL  CONCLUSION  about  the  organisms  which  require 
therapy. 
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{T he  tutor  has  begun  consideration  of  the  top  goal:  determining  a  regimen  of 
antibiotics  for  the  patient.  T he  student  has  never  used  the  program  before,  so  he  is 
shown  the  goal  rule  and,  next,  a  representation  of  the  AN  DIOR  tree  that  shows  only 
" important "  subgoals.  This  orientation  has  been  omitted  here  and  moved  to  Appendix 
D.\ 


We  will  now  begin  to  discuss  this  case,  limiting  our  discussion  to  one  topic 
at  a  time. 


{The  tutor  observes  that  this  topic  corresponds  to  the  goal  rule,  and  the  student  has 
never  discussed  it  before,  so  it  will  be  discussed  clause-by-clause... .  1 


(Now  considering  RULE092.) 

We  have  already  determined  that  information  has  been  gathered  about  organisms 
isolated  from  J.Smith,  organisms  noted  on  smears  of  J.Smith,  negative  cultures  of 
J.Smith,  suspected  infections  without  microbiological  evidence,  current  drugs 
of  J.Smith  and  prior  drugs  of  J.Smith. 


{7" he  procedure  for  discussing  a  rule  clause-by-clause  indicates  whether  a  clause  is 
done  or  not.  If  not,  the  procedure  for  discussing  a  goal  is  invoked  ) 

We  must  now  determine  whether  an  attempt  has  been  made  to  deduce  the 
organisms  which  require  therapy. 

We  ore  not  able  to  make  a  conclusion  about  the  organisms  which  require 
therapy,  because  there  are  no  organisms  In  this  case. 


[One  goal  is  to  determine  which  organisms  from  positive  cultures  require  therapy 
(some  may  be  contaminants).  The  tutor  notes  that  there  is  nothing  to  discuss  here 
because  we  have  no  organisms  to  consider.  T his  patient  will  be  treated  on  the  basis  of 
clinical  data  (social  history,  age,  etc.)  alone.) 

(Continuing  our  discussion  of  the  therapeutic  regimen  of  J.Smith  [RULE082].) 

We  must  now  determine  whether  you  have  given  consideration  to  organisms 
that  might  be  present,  and  found  at  least  one  organism  which  requires 
therapy. 
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{The  main  topic  of  the  tutorial  has  begun.  A  heuristic  based  on  the  number  of  rules 
associated  with  this  topic  indicates  that  it  is  important,  so  orientation  for  pursuing  it 
fj  presented:  a  definition,  list  of  important  subgoals,  indication  of  the  extent  of 
MYCIN’s  knowledge,  and  a  list  of  the  kinds  of  answers  to  look  for.} 

You  should  now  proceed  to  ask  questions  that  will  enable  you  to  make  an 
hypothesis  about  the  organisms  that  might  be  causing  the  infection. 

A  pathogenic  organism  which  was  not  seen  on  a  culture  or  smear  but 
which  might  be  causing  an  infection  in  the  patient  and  therefore  requires 
antimicrobial  therapy  Is  an  organism  to  "cover  for." 


•^Important 

factors  for  a 
major  topic 


CULTURE- 1  is  a  pending  culture.  In  this  context,  when  we  are 
considering  the  organisms  that  might  be  causing  the  infection,  we 
generally  find  it  useful  to  consider: 

8a.  the  Infection  which  requires  therapy 
8b.  the  type  of  the  infection 

8c.  whether  organisms  were  seen  on  the  stain  of  the  culture 
8d.  whether  a  smear  of  the  culture  was  examined 

There  are  03  rules  used  by  MYCIN  to  conclude  about  this  topic. 
Altogether  there  are  29  factors  considered  In  these  rules. 

Some  sample  values  for  the  organisms  that  might  be  causing  the  infection 
are:  proteus-non-mlrabilis,  streptococcus-group-b,  proteus-mirabilis  and 
neisseria-menlngitidis,  etc. 
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{The  student  begins  by  asking  for  information  about  the  ORIENT  option.  After 
using  it,  he  asks  to  be  quizzed.  Options  available  to  the  student  are  described  tn 
Chapter  7  (see  Figure  7.1  and  Appendix  C).} 

"»  HELP  ORIENT 

The  tutor  recites  all  of  the  topics  that  are  "on  the  stack." 

Starting  with  "discussing  a  case,"  it  moves  down  through  the  topics  and 
rules  that  have  been  mentioned,  to  explain  how  the  dialogue  got  to  its 
current  point.  Finally,  if  possible,  it  will  indicate  the  NEXT  steps 
that  will  be  taken  by  the  tutor.  Equivalent  to  typing  control-F. 


"LJSfacA  of  topic* 


{ The  history  of  active  topics  and  tutoring  rules  is  read  to  the  student.  T~rulel.0i  is 
retrieved  from  the  source  file  to  describe  what  will  be  done  next.} 

«*  ORIENT 

Discussing  a  case  with  the  student. 

(Applying  tutorial  rule  3.01). 

->  Discussing  the  therapeutic  regimen  of  J.Smith. 

(Applying  tutorial  rule  1 .04). 

Discussing  rule092  as  it  relates  to  finding  out  about  the 
therapeutic  regimen  of  J.Smith. 

->  Discussing  the  organisms  that  might  be  causing  the 
infection. 

The  remaining  steps  In  tutorial  rule  1 .04  are; 

Substep  v.  Say:  goodbye 

Substep  vl.  Record  that  the  goal  of  the  consultation  has  been  mentioned 
in  the  dialogue 

Substep  vii.  Exit  this  procedure 


{No  rules  for  this  topic  can  be  configured  into  one  of  the  available  question  formats. 
No  rules  have  succeeded  yet;  the  rules  that  failed  have  incomplete  subgoals .} 

**  QUIZ 

Sorry,  no  quizable  rules  have  been  determined  to  fail  or  succeed,  based  on 
information  available  to  you.  Request  HELP  to  discuss  other  rules. 


The  Protocol 


49 


<t|  First 

assistance  Is 
list  of 
factors  to 
consider 


**  HELP 


The  following  factors  will  be  useful  to  determine  the  organisms  that  might 
be  causing  the  Infection: 

9a.  whether  the  infection  was  acquired  while  the  patient  was 
hospitalized 

9b.  whether  J.Smith  has  a  head  injury  defect 
9c.  whether  J.Smith  has  a  tb  risk  factor 

9d.  whether  the  results  of  a  recent  CBC  of  J.Smith  are  available 
9e.  the  white  count  from  J.Smith's  peripheral  CBC  (In  thousands) 

9f.  whether  J.Smith  has  been  seriously  burned 
9g.  whether  J.Smith  has  sickle  cell  disease 
9h.  whether  J.Smith  has  had  neurosurgery 
9i.  whether  J.Smith  has  had  a  severe  head  trauma 
(*)  9j.  organisms  isolated  from  J.Smith 
9k.  the  infection  which  requires  therapy 
91.  the  type  of  the  infection 
9m.  whether  J.Smith  is  a  compromised  host 
(*)  9n.  the  race  of  J.Smith 

9o.  whether  J.Smith  has  evidence  of  serious  skin  or  soft  tissue 
infection 

9p.  whether  organisms  were  seen  on  the  stain  of  the  pending  csf 
culture 

9q.  whether  J.Smith  has  evidence  of  ocular  nerve  dysfunction 
9r.  whether  J.Smith  has  undergone  recent  radiation  therapy  to  the  CNS 
(*)  9s.  the  age  of  J.Smith 

9t.  whether  there  is  evidence  of  congenital  malformation  Involving 
the  central  nervous  system 
9u.  the  diagnoses  of  J.Smith 
9v.  whether  J.Smith's  X-ray  is  abnormal 
9w.  whether  J.Smith  does  live  In  a  crowded  environment 
9x.  whether  J.Smith  is  an  alcoholic 

9y.  the  epidemic  diseases  to  which  J.Smith  has  been  exposed  recently 
9z.  the  types  of  rash  which  J.Smith  has 

9aa.  whether  J.Smith  has  received  corticosteroids  just  prior  to  or 
during  this  infection 

9ab.  the  diagnoses  which  are  consistent  with  J.Smith's  clinical 
history 

9ac.  whether  J.Smith  has  had  a  splenectomy 

Factors  marked  with  '*'  have  already  been  stated.  Based  on  what  we  have 
already  mentioned,  you  should  be  able  to  conclude  about:  9p. 
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(This  list  is  produced  by  the  FACTORS  option.) 


{The  first  time  help  is  requested  for  a  topic,  the  tutor  invokes  the  FACTORS  option. 
Note  that  factor  9 p  is  knoum  to  MYCIN.  The  student  can  make  an  hypothesis,  say 
that  he  knows  it,  request  a  detailed  discussion,  or  ask  for  the  answer.  Here  he 
decides  to  offer  an  hypothesis  about  the  current  topic.) 


XX 

What  are  the  organisms  that  might  be  causing  the  infection? 

«*  NEISSERIA-MENINGITIS  (5) 

=NEISSERIA-MENINGITIDIS 


{ MYCIN  has  not  made  this  conclusion,  so  the  student  is  asked  to  support  his 
(wrong)  hypothesis.) 

What  facts  about  this  case  tell  you  that  Neisseria-meningitidis  Is  one  of  the 
organisms  that  be  causing  the  infection  (.6)? 

(Enter  one  factor  per  line.) 

**  AGE 

It* 


None  of  the  factors  that  you  mentioned  are  appropriate-. 

->  The  age  of  J. Smith  is  not  evidence  for  the  conclusion  that  Neisseria- 
meningitidis  is  one  of  the  organisms  that  might  be  causing  the  infection  (.&). 
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*OKey  factor • 
and  context 


{ The  tutor  points  out  that  the  age  of  the  patient  will  eventually  enable  the  student  to 
hypothesize  several  possible  organisms — it’s  just  premature  to  do  so  now.  Observe 
that  only  ' key  factors'  of  rules  are  printed  ( whether  J. Smith  has  a  head  injury 
defect )  and  that  the  " context "  of  these  conclusions  (two  rules)  depends  on  the 
infection  and  its  type.  GUIDON  has  in  fact  given  away  too  much  information  at 
this  point.} 

After  gaining  more  Information  about  this  case,  the  age  of  J. Smith  will  enable 
us  to  conclude  that  the  organisms  that  might  be  causing  the  infection  are: 
e.coli  (considering  whether  the  Infection  was  acquired  while  the 
patient  was  hospitalized  [RULE545]), 
pseudomonas-aeruginosa  (considering  [RULE545]), 
klebsiella-pneumoniae  (considering  [RULE645]), 

diplococcus-pneumoniae  (considering  whether  J. Smith  has  a  head  injury 
defect  and  the  age  of  J. Smith  [RULE507]), 
neisseria-meningitidis  (considering  [RULE507]), 
staphylococcus-coag-pos  (considering  [RULE545]). 

Remember  that  both  decisions  are  made  given  that:  1)  The  Infection  which 
requires  therapy  is  meningitis,  2)  Organisms  were  not  seen  on  the  stain  of 
the  pending  csf  culture  and  3)  The  type  of  the  Infection  is  bacterial. 


[After  the  hypothesis,  a  procedure  that  prepares  for  return  to  the  data  collection  mode 
is  followed.  The  student  decides  to  explore  the  topic  in  more  detail.) 

Note:  From  case  data  we  have  not  mentioned,  you  will  find  evidence  for: 
diplococcus-pneumoniae,  klebsiella-pneumoniae,  pseudomonas-aeruginosa  and 

e.coli. 

(Use  the  CONCLUDE  option  for  details.) 

Would  you  like  to  end  discussion  of  this  topic  now  (i.e.,  get  the  chance  to 
see  a  summary  of  all  evidence  that  can  eventually  be  collected  instead  of 
working  It  out  yourself)? 

**  NO 


[The  student  asks,  "What  rashes  does  this  patient  have?"  The  tutor  observes  that 
MYCIN  never  ashed  about  specific  rashes,  and  gives  the  student  the  more  general 
information  that  kept  MYCIN  from  asking.) 


« RASH 
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J.Smith  does  not  have  a  rash  or  cutaneous  lesions. 


[As  before,  MYCIN  rules  are  fired  as  new  information  is  given  to  the  student.  Thus 
a  record  is  kept  of  hypotheses  that  are  consistent  with  information  the  student  has 
received.} 

>>>  10a.  MYCIN  has  made  a  FINAL  CONCLUSION  about  whether  J.Smith  has 
evidence  of  serious  skin  or  soft  tissue  Infection. 

>>>  1 1  a.  MYCIN  has  made  a  FINAL  CONCLUSION  about  whether  J.Smith  has  a  rash 
consistent  with  Herpes  Zoster. 

>>>  12a.  MYCIN  has  made  a  FINAL  CONCLUSION  about  whether  J.Smith  has  had 
recent  vesicular  eruptions. 

>>>  1  da.  MYCIN  has  made  a  FINAL  CONCLUSION  about  the  types  of  rash  which 
J.Smith  has. 
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{The  student  wants  to  find  out  why  MYCIN  ultimately  decided  that  the  infection 
was  caused  by  bacteria  (page  51)  topic.  However,  kis  first  few  questions  (about 
neurosurgery,  x-ray,  and  hospital-acquired  infection)  reflect  continued  interest  in  the 
previous  topic,  the  organisms  causing  the  infection.  Finally,  he  inquires  about  about 
a  relevant  factor  (CSF  glucose ),  and  gets  enough  information  to  conclude  that  the 
infection  was  caused  by  bacteria.) 

**  DISCUSS  TYPE 

You  should  now  proceed  to  ask  questions  that  will  enable  you  to  make  an 
hypothesis  about  the  type  of  the  Infection. 


'^Definition  of  m 
topic 


An  infection  Is  classified  according  to  the  type  of  organism  causing 
the  infection  (e.g.  Bacteria,  Fungus,  Virus,  Tuberculosis). 


{ Orientation  is  presented  for  this  important  topic.) 


CULTURE- 1  Is  a  pending  culture.  In  this  context,  when  we  are 
considering  the  type  of  the  Infection,  we  generally  find  it  useful  to 
consider: 

1 4a.  the  infection  which  requires  therapy 

There  are  23  rules  used  by  MYCIN  to  conclude  about  this  topic. 

Altogether  there  are  20  factors  considered  in  these  rules. 

The  values  for  the  type  of  the  infection  are:  bacterial,  fungal,  viral  and  tb. 


{The  student  asks,  “Has  the  patient  had  neurosurgery?"  (from  page  49).  But 
GUIDON  doesn’t  want  to  discuss  neurosurgery  because  It  is  a  parameter  that 
shouldn’t  be  asked  until  rules  (and  other  factors)  are  considered  that  might  rule  it 
out.  T he  program  discovers  that  only  tht  "screening  question"  about  surgery  needs  to 
be  considered,  so  it  gives  this  information  to  the  student  and  proceeds  to  answer  his 
original  question.) 


**  9H 


J.Smlth  has  undergone  surgery. 
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It  is  not  known  whether  J.Smith  has  had  neurosurgery. 


TJSfucfenf  begin s 
to  jump 
around 


{MYCIN’s  pattern-matching,  keyword  parser  is  used  by  GUIDON.  In  this  case,  it 
can’t  tell  which  parameter  is  being  referred  to.) 

**  HAS  THE  PATIENT  HAD  AN  X-RAY? 

Sorry,  can't  parse  that. 

The  question  appeared  to  use  the  DATA  option,  In  reference  to  PATIENT-538. 

You  may  find  it  easier  to  use  the  single  command  format. 

Type  ?  for  assistance. 


*x|  Response  to 
irrelevant 
question 


{The  student's  question  is  related  to  the  current  topic.) 

*«  9V 

Whether  J. Smith's  X-ray  is  abnormal  is  not  useful  for  determining  the  type  of 
the  infection.  It  is  a  consideration  we  can  use  later  when  we  return  to  our 
discussion  of  the  organisms  that  might  be  causing  the  infection. 

J. Smith's  X-ray  Is  not  abnormal. 

**  9A 

Whether  the  infection  was  acquired  while  the  patient  was  hospitalized  is  not 
useful  for  determining  the  type  of  the  infection.  It  is  a  consideration  we 
can  use  later  when  we  return  to  our  discussion  of  the  organisms  that 
might  be  causing  the  infection. 

The  infection  was  acquired  while  the  patient  was  hospitalized. 


Typical  valid 
data  request 


••  CSF  GLUCOSE 
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The  csf  glucose  value  is  6. 

The  "CSF  findings"  are  a  set  of  measurements  that  are  usually 
available  after  a  lumbar  puncture  has  been  performed.  They  Include: 
the  CSF  protein,  the  W0C  from  the  CSF,  the  percent  of  PMN'a  In  the  CSF, 
and  the  CSF  glucose  value. 

The  CSF  findings  we  have  not  yet  mentioned; 

The  WBC  from  the  CSF  is  100. 


{MYCIN  rules  are  applied  as  neui  case  data  comes  in.  The  tutor  concludes  here 
( silently )  that  the  student  doesn't  know  any  of  these  " final  conclusions *  (I6a-18a).} 

>>>  15a.  MYCIN  just  made  a  conclusion  about  the  infection  which  requires 
therapy. 

>>>  16a.  MYCIN  has  made  a  FINAL  CONCLUSION  about  the  infection  which 
requires  therapy. 

>>>  17a.  MYCIN  has  made  a  FINAL  CONCLUSION  about  whether  the  meningitis  is 
partially  treated. 

>»  18a.  MYCIN  just  made  a  conclusion  about  the  type  of  the  Infection. 


{The  csf  glucose  is  one  of  a  block  of  related  test  results,  so  the  tutor  gives  all  of  them. 
By  requesting  details  about  any  numbered  remark  (e.g.J9a)  the  student  can  find  out 
what  MYCIN’s  conclusions  are.) 

♦ 

The  percent  of  pmns  in  the  csf  is  99. 

>>>  19a.  MYCIN  just  made  a  conclusion  about  the  type  of  the  Infection. 

The  CSF  protein  is  280. 

>>>  20a.  MYCIN  just  made  a  conclusion  about  the  type  of  the  infection. 

The  simultaneous  blood  glucose  value  Is  not  known. 

>>>  21a.  MYCIN  just  made  a  conclusion  about  the  type  of  the  infection. 

(Other  factors  in  this  block  have  already  been  given.) 


i 


|n. 
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{With  all  of  this  information,  MYCIN’s  currtnt  hypothesis  is  'close'  to  the  final 
answer  it  will  deduct.  To  save  time,  the  student  accepts  the  offer  to  wrap  up 
discussion.} 

We  have  substantially  completed  our  discussion  of  the  type  of  the  Infection. 

Would  you  like  to  make  a  hypothesis  now  (as  opposed  to  gathering  more 
information)? 

**  YES 


Ufttsponss  to 
hypothssls 


{The  student  makes  an  hypothesis  and  GUIDON  responds.} 

What  is  the  type  of  the  infection? 

**  BACTERIAL  (9) 


{T he  student’s  hypothesis  is  correct,  but  the  model  indicates  that  he  may  have  missed 
some  relevant  evidence.  Various  quiz  options  are  considered,  but  the  default  (t- 
rule25.05)  was  used:  ask  for  relevant  factors  to  support  the  hypothesis.) 

What  facts  about  this  case  tell  you  that  the  type  of  the  infection  is 
bacterial  (.9)? 

(Enter  one  factor  per  line.) 

*«  CSFGLUC 
PMNS 

The  percent  of  PMN's  In  the  CBC  Is  not  relevant  for  determining  the  type  of 
the  Infection. 

**  CSFPOLY 
**  PROTEIN 

*A 


•\JEvldencm 

summary 


{ Each  factor  is  checked  for  relevance  to  the  current  topic,  then  the  tutor  compares  the 
student’s  set  of  factors  to  those  mentioned  in  all  of  the  MYCIN  rules  that  cutrently 
support  the  hypothesis.) 

You're  right  about  the  CSF  protein,  the  percent  of  PMNs  in  the  CSF  and  the 
CSF  glucose  value,  however  you  should  have  mentioned  the  fact  that  the 
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meningitis  is  partially  treated  (.95),  the  WBC  from  the  CSF  and  the  fact  that 
the  simultaneous  blood  glucose  value  is  not  known. 

To  summarize,  we  can  conclude  that  the  type  of  the  infection  is: 
bacterial  (considering: 

whether  the  meningitis  is  partially  treated  [RULE  148], 
the  WBC  from  the  CSF  and  the  percent  of  PMNs  In  the  CSF 
[RULE501  ], 

the  CSF  protein  [RULE500]  and 

the  CSF  glucose  value  and  the  simultaneous  blood  glucose 
value  [RULE602]). 

Remember  that  these  decisions  are  made  given  that  the  infection  which 
requires  therapy  is  meningitis. 


MYCIN  rejected  some  possibilities  you  did  not  mention: 


{There  is  more  than  one  rule  that  supports  the  rejected  hypothesis  (viral),  so  again 
the  student  is  asked  to  list  relevant  factors  (t-rule26.0l).) 

What  facts  about  this  case  tell  you  that  the  type  of  the  Infection  is  not 
viral  (.87)7 

(Enter  one  factor  per  line.) 

«•  CSFGLUC 
**  CSFPOLY 
”  PROTEIN 


*Tl  Missing 
factor m 


{Observe  that  the  student  again  forgets  to  mrnffon  the  WBC  from  the  CSF  and  the 
simultaneous  glucose  value.  The  student  model  is  again  modified  to  show  that  he 
doesn’t  know  these  rules  (501  Id  502).) 

You're  right  about  the  CSF  protein,  the  percent  of  PMNs  in  the  CSF  and  the 
CSF  glucose  value,  however  you  should  have  mentioned  the  WBC  from  the  CSF  and 
the  fact  that  the  simultaneous  blood  glucose  value  is  not  known. 

To  summarize,  we  can  conclude  that  the  type  of  the  Infection  is: 
not  viral  (considering: 

the  WBC  from  the  CSF  and  the  percent  of  PMNs  in  the  CSF 
[RULE501], 
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the  CSF  protein  [RULE500]  and 

the  CSF  glucose  value  and  the  simultaneous  blood  glucose 
value  [RULE502]). 

Remember  that  these  decisions  are  made  given  that  the  Infection  which 
requires  therapy  is  meningitis. 


{The  remaining  rejected  values  are  concluded  by  a  single  rule  that,  a  priori,  is 
considered  to  be  more  than  one  level  above  the  student's  sophistication,  so  these 
conclusions  were  stated  directly  (t-rule26.02).  This  estimation  of  the  student's 
knowledge  is  borne  out  by  the  interaction  above  in  which  the  student  twice  neglects  to 
refer  to  rulebOI.} 

The  WBC  from  the  CSF  and  the  percent  of  PMNs  In  the  CSF  is  evidence  that  the 
type  of  the  infection  is  not  tb  (.7)  [RULE501]. 

The  WBC  from  the  CSF  and  the  percent  of  PMNs  in  the  CSF  is  evidence  that  the 
type  of  the  infection  Is  not  fungal  (.7)  [RULE601]. 


"^Detecting  topic 
term!  ft  ‘  nn 


It  remains  to  collect  further  evidence  for  determining  the  type  of  the 
infection. 

However,  we  have  substantially  completed  our  discussion  of  the  type  of  the 
infection. 

Would  you  like  to  end  discussion  of  this  topic  now  (i.e.,  get  the  chance  to 
see  a  summary  of  all  evidence  that  can  eventually  be  collected  instead  of 
working  it  out  yourself)? 

*•  NO 

Would  you  like  to  see  a  summary  of  the  evidence  we  have  discussed  before 
going  on  to  ask  more  questions? 

**  NO 
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{The  topic  continues  to  be  the  type  of  the  infection.  GUIDON  considers  quizzing 
about  two  rules  that  recently  failed  (but  they  are  too  advanced  for  the  student),  and 
four  rules  that  recently  succeeded  (but  they  were  just  discussed  above).  Nothing  is 
said.} 


•fj/ndirectly 

useful 

question 


{The  student’s  question  about  exposure  to  contagious  diseases  implicitly  brings  into 
focus  a  deeper  subgoal,  the  actual  diseases  to  which  the  patient  has  been  exposed.  (If 
the  patient  had  been  exposed  to  mumps,  one  could  conclude  that  the  type  of  the 
infection  is  viral.)  Rather  than  opening  up  discussion  of  the  deeper  subgoal,  the  tutor 
states  the  only  relevant  rule,  one  that  is  probably  known  by  the  student  (rule463). 
Compare  the  dialogue  here  to  what  occurred  when  abnormal  x-ray,  an  * irrelevant " 
question,  was  asked  on  page  54.} 

**  EXPOSED 

J. Smith  has  not  been  recently  exposed  to  a  contagious  disease. 

>»  22a.  MYCIN  has  made  a  FINAL  CONCLUSION  about  the  diseases  to  which 
J. Smith  has  been  exposed. 

>>>  23a.  MYCIN  has  made  a  FINAL  CONCLUSION  about  the  epidemic  diseases  to 
which  J.Smlth  has  been  exposed  recently. 

>>>  24a.  MYCIN  just  made  a  conclusion  about  the  organisms  (other  than  those 
seen  on  cultures  or  smears)  which  might  be  causing  the  infection. 

Your  question  is  indirectly  useful  for  determining  the  type  of  the  infection. 

whether  J.Smith  has  been  recently  exposed  to  a  contagious  disease  can  be 
used  to  determine  the  diseases  to  which  J.Smith  has  been  exposed... 

and  this  will  enable  us  to  determine  the  type  of  the  Infection. 

The  fact  that  J.Smith  has  not  been  recently  exposed  to  a  contagious  disease  is 

evidence  that  mumps  is  not  one  of  the  diseases  to  which  J.Smith  has  been 

exposed  [RULE463]. 

Back  to  our  discussion  of  the  type  of  the  infection.... 


i 

i 

k 


L 


{T he  student  now  requests  that  this  topic  be  terminated.) 
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«*  TELLME 

Our  final  conclusion  is  that  the  type  of  the  infection  is  bacterial  (.98) 
viral  (-.76)  fungal  (-.83)  tb  (-.83). 

Would  you  like  to  see  a  summary  of  the  evidence  that  supports  this 
conclusion? 

*•  YES 


{MYCIN’s  final  conclusion  is  not  tht  same  as  the  conclusion  that  can  be  made  from 
the  case  data  mentioned  so  far  in  the  dialogue,  so  first  GUIDON  lists  the  missing 
relevant  data.} 


Factors  that  have  not  been  mentioned  that  are  relevant  to  determining  the 
type  of  the  Infection: 

The  white  count  from  J.Smith's  peripheral  CBC  (in  thousands)  is  1 .9. 

>>>  26a.  MYCIN  just  made  a. conclusion  about  the  type  of  the  infection. 

J.Smith  has  had  recent  neurologic  signs. 

The  duration  of  the  neurological  signs  is  7.2  hours. 

>>>  26a.  MYCIN  just  made  a  conclusion  about  the  type  of  the  infection. 

J.Smith  has  had  recent  neurologic  symptoms. 

The  duration  of  the  neurological  symptoms  is  1  day. 1 

>»  27a.  MYCIN  just  made  a  conck  ion  about  the  type  of  the  infection. 

It  is  not  known  whether  lymphoma  Is  one  of  the  diagnoses  of  the  patient. 

It  is  not  known  whether  leukemia  is  one  of  the  diagnoses  of  the  patient. 

J.Smith  has  not  shown  symptoms  of  mumps. 

J.Smith  does  not  have  a  tb  risk  factor. 

»>  28a.  MYCIN  just  made  a  conclusion  about  the  organisms  that  might  be 
causing  the  infection. 
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{ Rules  that  made  conclusions  about  this  topic  art  now  described  in  terms  of  their  key 
factors.) 

Summary  of  evidence  that  the  type  of  the  infection  is  bacterial  (.98)  vlrai 

(-.76)  fungal  (-.83)  tb  (-.83): 

29a.  The  fact  that  the  meningitis  is  partially  treated  (.96)  is  evidence 
that  the  type  of  the  infection  is  bacterial  (.19)  [RULE148] 

29b.  The  WBC  from  the  CSF  and  the  percent  of  PMNs  in  the  CSF  Is  evidence 
that  the  type  of  the  infection  is  bacterial  (.9)  viral  (-.6)  fungal 
(-.7)  tb  (-.7)  [RUIE601] 

29c.  The  CSF  protein  is  evidence  that  the  type  of  the  infection  is  bacterial 
(.4)  viral  (-.6)  fungal  (.4)  tb  (.4)  [RULE600] 

29d.  The  csf  glucose  value  and  the  fact  that  the  simultaneous  blood  glucose 
value  is  not  known  is  evidence  that  the  type  of  the  infection  is 
bacterial  (.35)  viral  (-.36)  [RULE502] 

29e.  The  white  count  from  J.Smith's  peripheral  CBC  (In  thousands)  and  the  WBC 
from  the  CSF  is  evidence  that  the  type  of  the  infection  Is  bacterial 
(.2)  viral  (.15)  fungal  (.1)  tb  (.1)  [RULE604] 

29f.  The  duration  of  the  neurological  signs  is  evidence  that  the  type  of  the 
infection  is  bacterial  (.2)  viral  (.2)  fungal  (-.4)  tb  (-.4)  [RULE624] 

29g.  The  duration  of  the  neuroiogicai  symptoms  is  evidence  that  the  type  of 
the  infection  Is  bacterial  (.2)  viral  (.2)  fungal  (-.6)  tb  (-.6) 

[RULE526] 

Remember  that  these  decisions  are  made  given  that  the  Infection  which 

requires  therapy  Is  meningitis. 

ft**  ft  ft  *  *  II  * 


tl Forced 

tmr  mi  notion 
of  topic 


{ GUIDON  now  marks  the  inference  and  discussion  record  to  show  that  this  goal  has 
been  completed.  MYCIN  proceeds  to  use  this  final  conclusion  (31a).  As  always,  the 
student  model  is  updated  according  to  whether  GUIDON  believes  that  the  student  is 
following  along.) 
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>>>  30a.  MYCIN  has  made  a  FINAL  CONCLUSION  about  the  type  of  the  infection. 

>>>  31a.  MYCIN  just  made  a  conclusion  about  the  organisms  that  might  be 
causing  the  infection. 


Back  to  our  discussion  of  the  organisms  that  might  be  causing  the 
infection.... 


'‘tJQu/z  and 

response  to 
incorrect 

answer 


{After  discussing  the  type  of  the  infection,  the  topic  pops  back  to  determining  the 
organisms  that  are  causing  the  infection.  Seven  rules  that  have  failed  are  now 
considered  for  a  quiz.  One  has  tuio  incomplete  subgoals;  three  are  too  advanced  for 
this  student.  The  remaining  three  rules  have  the  same  ’interestingness’  score,  so  the 
choice  is  random.  The  premiselaction  format  of  the  question  is  'multiple  choice 
premise’l’ given  correct  conclusion."  (Possible  question  formats  are  summarized  in 
Appendix  B  ) 

Given  the  infection  which  requires  therapy,  what  value  for  the  type  of  the 
infection  would  tell  you  that  the  organisms  that  might  be  causing  the 
infection  are:  Cryptococcus  (1.0)  Coccidioides  (1.0)7 
(select  from  the  following): 

1)  VIRAL 

2)  FUNGAL 

3)  TB 

4)  BACTERIAL 

**  3 


No,  you  could  make  this  conclusion  if  the  type  of  the  Infection  were  fungal 
[RULE 666]. 
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*T|Student's 

attempt  to 
follow 
MYCIN’s 
reasoning 


{The  student  now  requests  mere  information  about  an  inference  about  the  previous 
topic.  He  will  then  examine  how  MYCIN  determined  a  subgoal  of  this  inference 
(that  the  infection  is  meningitis):  he  says  that  he  knows  it,  prints  the  rule  that 
MYCIN  used,  asks  for  a  justification  for  the  rule,  requests  to  discuss  the  subgoal, 
prints  relevant  factors  for  the  subgoal,  and  examines  a  rule  that  didn't  apply  to  this 
case. } 

**  DETAILS  29A 

For  which  of  the  following  factors  would  you  like  to  see  a  review  of  the 
evidence? 

(select  from  the  following): 

1 )  the  infection  which  requires  therapy 

2)  the  fact  that  the  meningitis  is  partially  treated  (.95) 

*«  2 


{GUIDON  observes  that  this  rule  which  it  mentioned  earlier  requires  two  subgoals 
that  were  not  discussed.  Consistent  with  the  student  model,  the  student  did  not  know 
how  to  determine  " partial  treatment.") 

Summary  of  evidence  that  the  meningitis  Is  partially  treated  (.95): 

32a.  The  infection  which  requires  therapy,  the  fact  that  organisms  were  not 
seen  on  the  stain  of  the  pending  csf  culture  (CULTURE-1)  and  the  time 
since  therapy  with  the  cephalothln  (DRUG-1)  was  started  Is  evidence 
that  the  meningitis  Is  partially  treated  (.95)  [RULE  145] 

Do  you  want  to  see  RULE  148? 

**  NO 


{The  student  tells  GUIDON  that  he  doesn’t  need  to  discuss  the  infection  that 
requires  therapy .} 

**  IKNOW  8A 

Good.  MYCIN  has  also  made  a  final  decision. 

The  Infection  with  which  the  pending  csf  culture  is  associated  is  evidence 
that  the  Infection  which  requires  therapy  Is  meningitis  [RULE333J. 


t 
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Back  to  our  discussion  of  the  organisms  that  might  be  causing  the 
infection.... 


{Rule333  is  a  complicated  way  of  saying  that  the  name  of  the  infection  derives  from 
the  body  location  that  is  infected .} 

**  PR  333 

RUUE333 


[This  rule  applies  to  cultures,  and  Is  tried  in  order  to  find  out  about  the 
infection  which  requires  therapy] 

If:  1 )  The  infection  which  requires  therapy  is  not  known, 

2)  The  infection  with  which  the  culture  is  associated  Is  known,  and 

3)  There  is  no  evidence  that  the  Infection  which  requires  therapy  is 
not  the  infection  with  which  the  culture  is  associated 

Then:  It  is  definite  (1.0)  that  the  infection  which  requires  therapy  is 
the  infection  with  which  the  culture  is  associated 
Author:  Scott  &  Shortllffe 
**  JUSTIFY  333 
[No  info] 


Discussing  a 
completed 
topic 


{The  student  decides  to  see  if  the  tutor  has  anything  further  to  say  about  the 
infection  that  requires  therapy.) 

**  CISCUSS  8A 

We  have  already  completed  our  discussion  of  the  infection  which  requires 
therapy. 

Our  final  conclusion  Is  that  the  infection  which  requires  therapy  Is 
meningitis. 

Would  you  like  to  see  a  summary  of  the  evidence  that  supports  this 
conclusion? 

*«  YES 


{ MYCIN  only  used  one  rule.) 
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Summary  of  evidence  that  the  Infection  which  requires  therapy  is  meningitis: 

33a.  The  infection  with  which  the  pending  csf  culture  Is  associated  Is 
evidence  that  the  infection  which  requires  therapy  is  meningitis 
[RULE333] 

*  *  ft  x  a  *  a  k  * 


Back  to  our  discussion  of  the  organisms  that  might  be  causing  the 
infection.... 


U  Displaying 
relevant 
factor m 


{ Still  not  satisfied,  the  student  asks  to  see  all  of  the  factors  for  concluding  the 
infection  which  requires  therapy.) 

"*  FACTORS  8A 

The  following  factors  were  useful  to  determine  the  infection  which  requires 
therapy: 

(*)  34a.  whether  J.Smith  has  had  recent  neurologic  symptoms 

(*)  34b.  whether  J.Smith  has  had  recent  neurologic  signs 

(*)  34c.  the  WBC  from  the  CSF 

(■)  34d.  the  CSF  protein 

(*)  34e.  the  site  of  CULTURE-1 

34f.  the  infection  with  which  the  pending  csf  culture  (CULTURE- 1 ) 
is  associated 

Factors  marked  with  have  already  been  stated.  Based  on  what  we  have 
already  mentioned,  you  should  be  able  to  conclude  about:  34f. 


{ Probing  further  to  understand  clause  3  of  rule} 33,  the  student  asks  for  details  for 
one  of  the  factors.  He  is  told  how  this  factor  might  have  changed  the  outcome  of  the 
case.) 

*"  DETAILS  34C 

If  J.Smith  had  not  had  recent  neurologic  symptoms,  J.Smith  had  not  had  recent 
neurologic  signs,  the  WBC  from  the  CSF  were  less  than  10  and  the  CSF  protein 
were  less  than  40,  one  could  conclude  that  the  infection  which  requires 
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therapy  is  not  meningitis  (.8)  [RULE1 16]. 

If  J.Smith  had  not  had  recent  neurologic  signs  and  the  WBC  from  the  CSF  were 
less  than  6,  one  could  conclude  that  the  infection  which  requires  therapy  is 
not  meningitis  (.7)  [RULE1 18]. 
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{Having  explored  a  topic  that  GUIDON  skipped  over  ( determine  the  name  of  the 
infection),  the  student  reorients  himself  to  tht  current  goal.} 


^Statement  of 
current  topic 


*«  TOPIC 

We  are  discussing  the  organisms  that  might  be  causing  the  infection. 

That  is,  you  should  be  considering  evidence  for  one  or  more  of  the  following: 
coccldioides,  bacteroides-fragilis,  hemophiius-influenzae  and  streptococcus- 
species,  etc. 


Student  begin * 
to  use 
handbook 


{At  this  point,  the  student  began  to  go  down  the  list  of  factors  that  appeared  in  the 
hardcopy  handbook  for  MYCIN.  He  asks  for  nine  pieces  of  data  in  this  series.) 

**  GRAM 

There  are  no  organisms  in  this  case,  so  "the  stain  of  the  organism"  is  not 
meaningful. 

*"  BURNED 

J.Smith  has  been  seriously  burned. 

>>>  36a.  MYCIN  just  made  a  conclusion  about  the  organisms  that  might  be 
causing  the  Infection. 


T|  Opportunistic 
tutoring 
Interruption 


{Here  an  "opportunistic  tutoring"  quiz  uses  the  format  of  "actual  premise 
facts' ("wrong  conclusion."  The  conclusion  is  chosen  randomly  from  other  MYCIN 
rules) 

Given  the  fact  that  J.Smith  has  been  seriously  burned,  does  this  tell  you  that 
Staphytococcus-coeg-pos  is  one  of  the  organisms  that  might  be  causing  the 
^faction  (  74)7 

••  m 
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That's  right  (see  RULE578). 

Given  these  facts,  we  can  conclude  that  Pseudomonas-aeruginosa  Is  one  of 
the  organisms  that  might  be  causing  the  Infection  (.49)  [RUIE578]. 

*«  NOSOCOMIAL 

You  were  already  given  this  Information... 

The  infection  was  acquired  while  the  patient  was  hospitalized. 


Hypothesis 
revision  hint 


**  HINT 

Hint:  Look  for  strong,  positive  evidence  that  Diplococcus-pneumonlae  is  one 
of  the  organisms  that  might  be  causing  the  infection. 


{Applying  an  hypothesis  revision  strategy  to  order  the  rules  that  remain  to  be 
considered,  GUIDON  suggests  that  the  student  consider  Diplococcus.  However,  the 
student  decides  to  look  more  closely  at  another  hypothesis....} 

"*  CONCLUDE  PSEUDOMONOS-ARIGINOSA 

I  don't  recognize  PSEUDOMONOS-ARIGINOSA  as  a  value  for  the  organisms  that 
might  be  causing  the  infection. 

(Type  control-0  to  abort  printout) 

Choose  from  ACINETOBACTER-ANITRATUS,  ACINETOBACTER-LWOFFI,  ACINETOBACTER- 
SPECIES,  ACTINOMYCES-ISRAELII,  ACTINOMYCES-OOONTOLYTICUS,  ACTINOMYCES- 
SPECIEStO 


•^IFacfors 

relevant  to 
an 

hypothesis 


{Describing  how  an  hypothesis  can  be  supported,  GUIDON  mentions  future 
conclusions,  evidence  that  has  not  been  discussed,  and  offers  to  list  discussed 
evidence.} 

*’  CONCLUDE  PSEUDOMONAS-AERUGINOSA 

Using  the  following  factors,  it  will  be  possible  to  conclude  that  Pseudomonas- 
aeruginosa  Is  one  of  the  organisms  that  might  be  causing  the  infection: 

36a.  the  white  count  from  J.Smith's  peripheral  CBC  (In  thousands) 
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[RULE557] 


Using  the  following  factors,  it  is  possible  to  conclude  that  Pseudomonas- 
aeruginosa  is  one  of  the  organisms  that  might  be  causing  the  infection: 

37a.  the  fact  that  the  infection  was  acquired  while  the  patient  was 
hospitalized  [RUI.E545] 


Do  you  want  information  about  evidence  that  we  have  already  discussed  (if 
any)? 

**  NO 


{ Returning  to  the  list  of  factors  in  the  handbook,  the  student  is  now  looking  for 
evidence  of  Diplococcus.  The  current  version  of  GUIDON  does  not  monitor  the 
student's  strategies.} 

**  CSRSUG 

You  were  already  given  this  information... 

J.Smith's  X-ray  is  not  abnormal. 
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Tl  Premature 
question 


{Adhering  strictly  to  MYCIN's  rules,  the  tutor  refuses  to  give  information  about 
alcoholism  until  the  student  has  considered  whether  the  patient  is  a  compromised 
host,  a  more  general  condition.  This  tutoring  rule  needs  to  be  modified.) 

**  ALCOHOLIC 

While  you  will  eventually  find  it  useful  to  ask  about  whether  J.Smith  is  an 
alcoholic,  It  is  premature  to  do  so  now.  There  is  still  a  possibility  that 
other  data  will  enable  you  to  deduce  a  value. 

**  CROWD 

J. Smith  does  not  live  in  a  crowded  environment. 


If  J.Smith  did  live  In  a  crowded  environment,  would  this  tell  you  that  Virus 
is  one  of  the  organisms  that  might  be  causing  the  Infection  (.89)7 
**  NO 

That's  right  (see  RULE533). 

If  J. Smith  did  live  in  a  crowded  environment,  one  could  conclude  that 
Neisseria-menlngitidis  is  one  of  the  organisms  that  might  be  causing  the 
infection  (.2)  [RULE633]. 


{ T he  student  asks,  " Does  the  patient  have  a  fever?"} 

**  FEBRILE 

MYCIN  never  needed  to  inquire  about  whether  J.Smith  is  febrile. 

Whether  J.Smith  is  febrile  is  not  relevant  to  our  discussion  of  the  organisms 
that  might  be  causing  the  Infection. 


{The  student  now  asks  about  a  splenectomy  because  it  is  evidence  for  diplococcus. 
T he  tutor’s  question  about  a  splenectomy  remind;  the  student  that  sickle  cell  disease 
was  also  evidence  for  diplococcus,  and  he  types  this  in  his  excitement  to  find  out 
about  sickle  cell  disease.  Then,  still  anxious  to  find  out  about  sickle  cell  disease,  he 
mistakes  the  summary  of  rule) 39  to  be  another  quiz,  and  types  T  In  agreement  with 
the  conclusion.) 

"  SPLENECTOMY 

J.Smith  has  not  had  a  splenectomy. 
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If  J. Smith  had  had  a  splenectomy,  would  this  tell  you  that  Streptococcus- 
group-b  is  one  of  the  organisms  that  might  be  causing  the  Infection  (.69)? 
»  SICKLECELL 
Yes,  No,  or  Dontknow,  please. 

-NO 

That's  right  (see  RULE559). 

If  J.Smith  had  had  a  splenectomy,  one  could  conclude  that  Diplococcus- 
pneumoniae  is  one  of  the  organisms  that  might  be  causing  the  Infection  (.6) 
[RUIE659]. 

**  Y 

Can't  make  sense  of:  Y. 

Enter  PARAMETER/CONTEXT  pair  or  index,  e.g.,  1 4a. 


( GUIDON  observes  that  MYCIN  never  asked  about  sickle  cell  disease,  and  states  the 
rule  (281)  that  gives  the  answer,  a  rule  which  the  student  almost  certainly  knows.) 

-  SICKLECELL 

The  race  of  J.Smith  is  evidence  that  J.Smith  does  not  have  sickle  cell  disease 
[RULE281]. 


{A  question  is  posed  using  the  format  is  " actual  fact  s' I"  correct  conclusion .“} 

Given  the  fact  that  J.Smith  does  not  have  sickle  cell  disease,  does  this  tell 
you  that  Oiplococcus-pneumoniae  is  one  of  the  organisms  that  might  be  causing 
the  infection  (.76)? 

”  YES 

That's  right  (see  RUIE531). 
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•\JRamaining 

factors 


{The  student  stops  to  see  what  remains  to  be  done.  This  leads  him  to  request  more 
data  and  to  investigate  why  certain  factors  are  relevant .} 

«■  PENDING 

We  need  to  discuss  the  following  subgoals  before  our  discussion  of  the 
organisms  that  might  be  causing  the  infection  will  be  complete: 

38a.  whether  J.Smith  has  received  corticosteroids  Just  prior  to  or 
during  this  infection 
38b.  whether  J. Smith  is  an  alcoholic 
38c.  the  diagnoses  of  J.Smith 

38d.  whether  there  is  evidence  of  congenital  malformation  Involving 
the  central  nervous  system 

38e.  whether  J.Smith  has  undergone  recent  radiation  therapy  to  the  CNS 
38f.  whether  J.Smith  has  had  a  severe  head  trauma 
38g.  whether  J.Smith  has  a  head  injury  defect 


We  need  to  discuss  the  following  case  data  that  is  directly  relevant  to  the 
organisms  that  might  be  causing  the  infection: 

39a.  the  diagnoses  which  are  consistent  with  J.Smith's  clinical 
history 

39b.  whether  J.Smith  has  evidence  of  ocular  nerve  dysfunction 

39c.  whether  J.Smith  is  a  compromised  host 

39d.  whether  the  results  of  a  recent  CBC  of  J.Smith  are  available 


You  should  be  able  to  make  an  hypothesis  about  the  following  subgoals: 

40a.  the  types  of  rash  which  J.Smith  has 

40b.  the  epidemic  diseases  to  which  J.Smith  has  been  exposed  recently 
40c.  whether  organisms  were  seen  on  the  stain  of  the  pending  csf 
culture 

40d.  whether  J.Smith  has  evidence  of  serious  skin  or  soft  tissue 
infection 


{ The  tutor’s  reasoning  for  presenting  information  about  a  skin  infection  is  the  same 
as  for  sickle  cell  disease....) 
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"  SKININFECT 

The  fact  that  J.Smith  does  not  have  a  rash  or  cutaneous  lesions  Is  evidence 
that  J.Smith  does  not  have  evidence  of  serious  skin  or  soft  tissue  infection 
[RULE369]. 


{No  attempt  has  been  made  to  document  all  of  MYCIN's  parameters  at  this  time....} 

••  DEFINE  DIAGNOSES 

Sorry,  there  is  no  entry  for  DIAGNOSES  in  MYCIN's  definition  file. 


{The  student  asks  to  discuss  the  diagnoses  that  are  consistent  with  the  patient’s 
clinical  history.  There  are  no  rules  to  discuss,  so  GUIDON  gioes  the  case  data  and 
moves  on.} 

”  DISCUSS  39A 

The  diagnoses  which  are  consistent  with  J.Smith's  clinical  history  is 
information  about  a  case  that  is  usually  available  to  the  consultant. 

Epiglottitis  is  not  one  of  the  diagnoses  which  are  consistent  with  the 
patient's  clinical  history.  • 

Otitis-media  Is  not  one  of  the  diagnoses  which  are  consistent  with  the 
patient's  clinical  history. 

>>>  41a.  MYCIN  Just  made  a  conclusion  about  the  organisms  that  might  be 
causing  the  Infection. 

Back  to  our  discussion  of  the  organisms  that  might  be  causing  the 
infection.... 


{This  quiz  format  is  ’ wrong  premise  hypothesis' /"correct  conclusion"....}  information 
about  a  case  that  is  usually  available  to  the  consultant. 

Given  the  infection  which  requires  therapy,  if  coccidioidomycosis  were  one  of 
the  diagnoses  which  are  consistent  with  J.Smith's  clinical  history,  would  this 
tell  you  that  Virus  is  one  of  the  organisms  that  might  be  causing  the 
infection  (.2)7 
**  NO 

That's  right  (see  RULE680). 

You  could  make  this  conclusion  if  epiglottitis  were  one  of  the  diagnoses 
which  are  consistent  with  J.Smith's  clinical  history  [RULE680]. 


««  PR  580 
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RULE580 


[This  rule  applies  to  all  cultures  and  suspected  infections,  and  is  tried 
in  order  to  find  out  about  the  organisms  that  might  be  causing  the 
infection] 

If:  1 )  The  infection  which  requires  therapy  is  meningitis, 

2)  Epiglottitis  Is  one  of  the  diagnoses  which  are  consistent  with  the 
patient's  clinical  history 

Then:  There  is  weakly  suggestive  evidence  (.2)  that  virus  is  one  of  the 
organisms  that  might  be  causing  the  infection. 

Author:  Yu 

**  39B 

J. Smith  does  not  have  evidence  of  ocular  nerve  dysfunction. 


{At  this  point,  GUIDON  decides  not  to  quiz  about  a  rule  that  just  failed  (366) 
because  it  is  too  advanced  for  the  student.) 

»«  USE  39B 

If  J. Smith  had  evidence  of  ocular  nerve  dysfunction,  one  could  conclude  that 
Mycobacterium-tb  Is  one  of  the  organisms  that  might  be  causing  the 
Infection  (.2)  [RULE366]. 


iJHIfty  a  factor 
relevant 


{ When  explaining  how  a  factor  is  used,  GUIDON  mentions  only  inferences  relevant 
to  the  current  topic.  For  example,  the  association  between  compromised  host  and 
alcoholism  is  not  mentioned  here.) 

•«  USE  39C 

J. Smith  Is  a  compromised  host. 

If  the  type  of  the  Infection  were  fungal,  J.Smlth  were  not  a  compromised  host 
and  the  race  of  J. Smith  were  one  of:  black  asian  Indian,  one  could  conclude 
that  Cryptococcus  is  not  one  of  the  organisms  that  might  be  causing  the 
Infection  (.6)  (other  Information  would  also  need  to  be  available  to  make 
this  conclus»on)[RULE280]. 

-  USE  390 
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The  results  of  a  recent  CBC  of  J.Smith  are  available. 

>»  42a.  MYCIN  just  made  a  conclusion  about  the  organisms  that  might  be 
causing  the  infection. 


\As  rules  like  557  are  presented,  the  student  model  is  updated  to  show  that  the 
student,  like  MYCIN,  may  use  the  stated  conclusions  in  other  rules  that  he  knows.} 

Using  other  information  available  to  you  now,  the  fact  that  the  results  of  a 
recent  CBC  of  J.Smith  are  available  enables  you  to  conclude  that  the  organisms 
that  might  be  causing  the  Infection  are:  Pseudomonas-aeruginosa  (.49) 

E.coli  (.74)  Klebsiella-pneumoniae  (.49)  [RULE5S7]. 

•»  FOLEY 

MYCIN  never  needed  to  inquire  about  whether  J.Smith  has  an  indwelling  urinary 
catheter. 

Whether  J.Smith  has  an  indwelling  urinary  catheter  is  not  relevant  to  our 
discussion  of  the  type  of  the  infection. 
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■tJ Economic  a  I 

presentation 


[The  discussion  that  follows  is  a  bit  complex.  The  student  asks  for  help  and  rule507 
is  chosen,  following  up  on  the  hint  ("look  for  evidence  of  Diplococcus-Pneumoniae") 
that  the  program  made  earlier  (page  68).  One  subgoal  remains  to  be  done  (head 
injury);  the  presentation  method  (t-rule3l.04)  is  to  tell  the  student  to  work  on  this 
sub  goal.  However,  GUIDON  observes  (belatedly!)  that  this  subgoal  can  be  deduced 
from  a  rule  that  student  probably  knows  (rule)09),  so  It  states  the  conclusion,  rather 
than  giving  him  the  initiative....} 

««  HELP 

Try  to  determine  whether  J.Smith  has  a  head  injury  defect. 

The  fact  that  J.Smith  has  not  had  an  injury  or  insult  to,  or  defect  in  the  CNS 

is  evidence  that  J.Smith  does  not  have  a  head  injury  defect  [RULE 509], 

»>  43a.  MYCIN  has  made  a  FINAL  CONCLUSION  about  whether  J.Smith  has  had  a 
severe  head  trauma. 

>>>  44a.  MYCIN  has  made  a  FINAL  CONCLUSION  about  whether  there  is  evidence 
of  congenital  malformation  involving  the  central  nervous  system. 

>>>  45a.  MYCIN  has  made  a  FINAL  CONCLUSION  about  whether  J.Smith  has  a  head 
injury  defect. 

»>  46a.  MYCIN  just  made  a  conclusion  about  the  organisms  that  might  be 
causing  the  infection. 


*rl Wrapping  up 
dfacuaaion 


{The  procedure  for  wrapping  up  discussion  of  a  rule  is  now  followed:  quiz  about  or 
state  the  conclusion,  summarize  cumulative  evidence,  report  on  the  status  of  the 
current  goal,  discuss  related  rules,  and  return....} 

We  can  now  conclude  that  the  organisms  that  might  be  causing  the  infection 
are:  Diplococcus-pneumoniae  (.74)  Neisserla-meningitidis  (.73)  (considering 
whether  J.Smith  has  a  head  injury  defect  and  the  age  of  J.Smith  [RULE607]). 

The  evidence  we  have  discussed  so  far  Indicates  that  the  organisms  that  might 
be  causing  the  infection  are:  E.coli  (.93)  Pseudomonas-aeruginosa  (.82) 
Klebslella-pneumoniae  (.74)  Diplococcus-pneumoniae  (.74) 
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Neisseria-meningitidis  (.45)  Staphylococcus-coag-pos  (.29)  Mycobacterium-tb 
(-.4). 

It  remains  for  us  to  consider  other  factors  for  determining  the  organisms 
that  might  be  causing  the  infection. 


*Q  Opportunlatlc 
tutoring 
Interruption 


{"Related  rules’  mention  one  or  more  of  the  hypotheses  concluded  by  the  rule  that  was 
just  discussed.) 

»»»»»»»»»»»»»»»»»»»»»»»»»»»»» 

>>>>>>>>>>  Considering  related  rules  >>>>>>>>>> 

»»»»»»»»»»»»»»»»»»»»»»»»»»»»» 


{This  quiz  format  is  "actual  fact  s' I" confirm  related  evidence."  Strictly  speaking,  only 
Neisseria  is  confirmed.) 

Does  information  about  the  types  of  rash  which  J.Smith  has  tend  to  confirm  the 
conclusion  that  the  organisms  that  might  be  causing  the  infection  are: 
Diplococcus-pneumoniae  (.74)  Neisseria-meningitidis  (.79)  [RUIE507]? 

**  NO 

That’s  right  (see  RULE539). 


{This  quiz  format  ( for  a  rule  that  fails)  is  "right  hypothesis"l"correct  conclusion") 

If  meningococcal-disease  were  one  of  the  epidemic  diseases  to  which  J.Smith 
has  been  exposed  recently,  would  this  tell  you  that  Neisseria-meningitidis  is 
one  of  the  organisms  that  might  be  causing  the  infection  (.8)7 
”  YES 

That's  right  (see  RULE637). 


{This  quiz  format  (for  a  rule  that  fails)  is  "right  premise  hypothesis" /’multiple 
choice  conclusion") 

If  otitis-media  were  one  of  the  diagnoses  which  are  consistent  with  J.Smlth’s 
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clinical  history,  one  could  conclude  that  the  organisms  that  might  be  causing 

the  infection  are 

(select  from  the  following): 

1)  staphylococcus-coag-pos  —  weakly  suggestive  evidence  (.29) 

2)  diplococcus-pneumoniae  --  suggestive  evidence  (.7) 

3)  pseudomonas-aeruginosa  —  strongly  suggestive  evidence  (.82) 

4)  e.coll  —  strongly  suggestive  evidence  (.93) 

5)  streptococcus-species  —  weakly  suggestive  evidence  (.09) 

6)  cryptococcus  --  weakly  suggestive  evidence  (.22) 

7)  not  peptostreptococcus  --  weakly  suggestive  evidence  (-.23) 

8)  neisseria-meningitidis  --  weakly  suggestive  evidence  (.45) 

9)  listeria  --  strongly  suggestive  evidence  (.82) 

1 0)  not  mycobacterium-tb  —  weakly  suggestive  evidence  (-.4) 

1 1)  klebsiella-pneumoniae  —  suggestive  evidence  (.74) 

12)  hemophilus-influenzae  —  suggestive  evidence  (.75) 

**  2  12 

Yes,  that's  right  (see  RULE549). 

»»»»»»»»»»»»»»>»»»»»»»»»»»»»»»> 
>>>>>>>>>>  End  of  our  discussion  of  related  rules  >>>>>>>>> 
>>>>»»>»>>>>>>>>>»>>»»»»»»»>»»>»»>»>»>>»» 

(End  of  our  discussion  of  a  rule  to  determine  the  organisms  that  might  be 
causing  the  infection  [RULE507].) 


We  have  substantially  completed  our  discussion  of  the  organisms  that  might 
be  causing  the  infection. 

Would  you  like  to  make  a  hypothesis  now  (as  opposed  to  gathering  more 
information)? 

**  NO 
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{The  student  returns  to  determining  the  organisms  that  might  be  causing  the 
infection.} 

«  CONCLUDE  E.COLI 

All  evidence  for  this  conclusion  has  already  been  discussed. 

Do  you  want  Information  about  evidence  that  we  have  already  discussed  (if 
any)? 

■«  YES 

Using  the  following  factors,  we  have  already  discussed  how  to  conclude  that 
E.coll  is  one  of  the  organisms  that  might  be  causing  the  Infection: 

47a.  the  white  count  from  J. Smith's  peripheral  CBC  (in  thousands) 

[RULE657] 

47b.  the  fact  that  the  infection  was  acquired  while  the  patient  was 
hospitalized  [RUIE646] 


**  PR  557 

RULE557 


[This  rule  applies  to  pending  cultures  and  suspected  infections,  and  is  tried 
in  order  to  find  out  about  the  organisms  that  might  be  causing  the 
infection] 

If:  1 )  The  infection  which  requires  therapy  Is  meningitis, 

2)  A:  A  smear  of  the  culture  was  not  examined,  or 

B:  Organisms  were  not  seen  on  the  stain  of  the  culture, 

3)  The  type  of  the  Infection  is  bacterial, 

4)  The  results  of  a  recent  CBC  of  the  patient  are  available,  and 

6)  The  white  count  from  the  patient's  peripheral  CBC  (in  thousands)  Is 
less  than  2.6 

Then:  There  is  evidence  that  the  organisms  that  might  be  causing  the 
Infection  is  pseudomonas-aeruginosa  (.5)  e.coli  (.76) 
klebsiella-pneumonlae  (.6) 

Author:  Yu 


{ Now  asked  to  ' explain '  this  rule,  GUIDON  can  account  for  the  contextual  clauses 
(1-3)  and  the  screening  clause  (4),  but  it  doesn't  know  how  to  fit  WBC  into  its  model 
of  sufficient  evidence  for  organisms  causing  an  infection.  It  doesn’t  know  that  this 
white  blood  count  range  is  termed  * leukopenia ’  and  that  such  a  condition  is 
nonspecific  evidence  of  an  infection.} 


«*  EXPLAIN  557 
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Can't  explain  the  clause: 

the  white  count  from  J. Smith's  peripheral  CBC  (in  thousands)  is  less  than 
2.5 


{Here  the  program  restates  the  student's  question  because  the  parse  is  ambiguous. 
T he  reply  shows  that  GUIDON,  like  any  good  human  tutor,  has  a  sense  of  humor,  j 

««  WHAT  IS  THE  PATIENT’S  WHITE  COUNT? 

[I.e.,  you  are  asking  about  the  race  of  J. Smith.] 

You  were  already  given  this  information... 

The  race  of  J. Smith  is  Caucasian. 

**  WHAT  IS  THE  PATIENT’S  WBC? 

You  were  already  given  this  information... 

The  white  count  from  J. Smith's  peripheral  CBC  (in  thousands)  is  1.9. 


{T he  student  is  tired  and  decides  to  call  it  quits.} 

**  TELLME 

Our  final  conclusion  is  that  the  organisms  that  might  be  causing  the 
infection  are:  E.coli  (.93)  Pseudomonas-aeruginosa  (.82) 
Klebsiella-pneumoniae  (.74)  Diplococcus-pneumoniae  (.74) 
Neisseria-meningitidis  (.45)  Staphylococcus-coag-pos  (.29)  Mycobacterium-tb 
(-.4)  Hemophilus-influen2ae  (-.4). 

Would  you  like  to  see  a  summary  of  the  evidence  that  supports  this 
conclusion? 

**  YES 


Factors  that  have  not  been  mentioned  that  are  relevant  to  determining  the 
organisms  that  might  be  causing  the  infection: 

J. Smith  is  not  an  alcoholic. 


Summary  of  evidence  that  the  organisms  that  might  be  causing  the  infection 
ar"  E.coli  (.93)  Pseudomonas-aeruginosa  (.82)  klebsiella-pneumoniae  (.74) 
Diplococcus-pneumoniae  (.74)  Neisseria-meningitidis  (.45) 
Staphylococcus-coag-pos  (.29)  Mycobacterium-tb  (-.4)  Hemophilus-influenzae 
(-.4): 
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48a.  The  infection  which  requires  therapy,  the  fact  that  the  infection  was 
acquired  while  the  patient  was  hospitalized  and  information  about  the 
epidemic  diseases  to  which  J.Smith  has  been  exposed  recently  is  evidence 
that  Neisseria-meningitidis  is  not  one  of  the  organisms  that  might  be 
causing  the  infection  (.5)  [RULE561] 

48b.  The  infection  which  requires  therapy,  the  fact  that  J. Smith's  X-ray  is 
not  abnormal  and  the  fact  that  J.Smith  does  not  have  a  tb  risk  factor 
is  evidence  that  Mycobacterium-tb  is  not  one  of  the  organisms  that 
might  be  causing  the  infection  (.4)  [RULE  160] 

48c.  The  fact  that  the  infection  was  acquired  while  the  patient  was 

hospitalized  is  evidence  that  the  organisms  that  might  be  causing  the 
infection  are:  E.coli  (.74)  Staphylococcus-coag-pos  (.29) 
Pseudomonas-aeruginosa  (.29)  Klebsiella-pneumoniae  (.49)  [RULE645] 

48d.  The  fact  that  J.Smith  has  been  seriously  burned  is  evidence  that 
Pseudomonas-aeruginosa  is  one  of  the  organisms  that  might  be 
causing  the  infection  (.49)  [RULE678] 

48e.  The  infection  which  requires  therapy,  the  fact  that  the  infection  was 
acquired  while  the  patient  was  hospitalized  and  information  about  the 
diagnoses  which  are  consistent  with  J.Smith's  clinical  history  is 
evidence  that  Hemophilus-influenzae  is  not  one  of  the  organisms  that  i 
might  be  causing  the  infection  (.4)  [RULE662] 

48f.  The  white  count  from  J.Smith's  peripheral  C8C  (in  thousands)  is  evidence 
that  the  organisms  that  might  be  causing  the  infection  are: 
Pseudomonas-aeruginosa  (.49)  E.coli  (.74)  Klebsiella-pneumoniae  (.49) 
[RULE557] 

48g.  The  fact  that  J.Smith  does  not  have  a  head  injury  defect  and  the  age  of 
J.Smith  is  evidence  that  the  organisms  that  might  be  causing  the 
infection  are:  Diplococcus-pneumoniae  (.74)  Neisseria-meningitidis 
(.73)  [RULE507] 

Remember  that  these  decisions  (RULE607,  RULE667,  RULE678  and  RULE545)  are 

made  given  that:  1)  The  infection  which  requires  therapy  is  meningitis,  2) 

Organisms  were  not  seen  on  the  stain  of  the  pending  csf  culture,  and  3)  The 

type  of  the  infection  is  bacterial. 


>»  49a.  MYCIN  has  made  a  FINAL  CONCLUSION  about  the  organisms  that  might  be 
causing  the  Infection. 

>>>  60a.  MYCIN  just  made  a  conclusion  about  the  organisms  that  will  be 
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considered  to  be  causing  the  meningitis  for  the  purposes  of  therapy 
selection. 

>>>  61a.  MYCiN  has  made  a  FINAL  CONCLUSION  about  the  organisms  that  will  be 
considered  to  be  causing  the  meningitis  for  the  purposes  of  therapy 
selection. 

(Continuing  our  discussion  of  the  therapeutic  regimen  of  J.Smith  [RULE092].) 


i|C/ause-6y- 

clause 

discussion 


{From  now  to  the  end  of  the  session,  the  student  gets  the  initiative  exactly  once.  T he 
tutor  methodically  completes  discussion  of  the  goal  rule....) 


We  must  now  consider  whether  the  organisms  Isolated  from  the  pending  csf 
culture  should  be  considered  for  therapy. 

Let's  consider  a  way  to  determine  this  by  definition.... 


(Now  considering  RULE049.) 

We  have  already  determined  that  the  infection  which  requires  therapy  is  known. 

We  have  already  determined  that  you  have  told  the  user  which  infection  will 
be  associated  with  this  culture  site. 

We  must  now  determine  whether  an  attempt  has  been  made  to  deduce  the  time 
since  clinical  evidence  of  the  infection  was  first  noted. 

There  are  no  rules  which  successfully  apply  to  this  case  that  can  tell  us 
about  the  time  since  clinical  evidence  of  the  Infection  was  first  noted. 

Since  it  is  not  deducible,  a  consultant  would  have  to  ask  to  see  if  the 
person  seeking  advice  knew  about  It. 

The  reported  case  data: 

The  time  since  clinical  evidence  of  the  infection  was  first  noted  is  1  day. 

>>>  62a.  MYCIN  just  made  a  conclusion  about  whether  the  organisms  Isolated 
from  the  pending  csf  culture  should  be  considered  for  therapy. 

>>>  63a.  MYCIN  has  made  a  FINAL  CONCLUSION  about  whether  the  organisms 
isolated  from  the  pending  csf  culture  should  be  considered  for 
therapy. 
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>>>  54a.  MYCIN  has  made  a  FINAL  CONCLUSION  about  the  number  of  cultures  from 
this  site  since  the  onset  of  the  Infection. 

(Continuing  our  discussion  of  whether  the  organisms  Isolated  from  the  pending 
csf  culture  should  be  considered  for  therapy  [RULE049].) 

We  can  now  conclude  that  the  organisms  isolated  from  the  pending  csf  culture 
should  be  considered  for  therapy  (considering  the  site  of  CULTURE*  1,  the 
infection  which  requires  therapy  and  the  time  since  clinical  evidence  of 
the  infection  was  first  noted  [RULE049]). 

However,  there  are  other  rules  that  we  might  discuss  that  use  information  you 
have  already  been  given. 

(End  of  our  discussion  of  a  rule  to  determine  whether  the  organisms  isolated 
from  the  pending  csf  culture  should  be  considered  for  therapy  [RULE049].) 


(Continuing  our  discussion  of  the  therapeutic  regimen  of  J.Smlth  [RULE092].) 

We  must  now  consider  the  associated  infections  for  which  therapy  should  cover. 
We  are  not  able  to  make  a  conclusion  about  the  associated  infections  for 
which  therapy  should  cover,  because  there  are  no  organisms  In  this  case. 

(Continuing  our  discussion  of  the  therapeutic  regimen  of  J.Smith  [RULE092].) 

We  must  now  consider  the  organisms  that  will  be  considered  to  be  causing  the 
meningitis  for  the  purposes  of  therapy  selection. 

Try  to  make  a  conclusion  about  the  organisms  that  will  be  considered  to  be 
causing  the  meningitis  for  the  purposes  of  therapy  selection. 

What  are  the  organisms  that  will  be  considered  to  be  causing  the  meningitis 
for  the  purposes  of  therapy  selection? 

**  DONTKNOW 


{The  student  didn’t  care  to  answer.  This  parameter  is  not  meaningful  in  this  case, 
for  it  simply  involves  transferring  results  from  one  intermediate  concept  to  another. 
If  there  were  positive  cultures,  this  clinical  evidence  would  have  to  he  reconciled  with 
likely  identities  of  organisms  growing  on  these  cultures .} 

Information  about  the  organisms  that  might  be  causing  the  infection  is 
evidence  that  the  organisms  that  will  be  considered  to  be  causing  the 
meningitis  for  the  purposes  of  therapy  selection  are:  e.coii  (.37) 
pseudomonas-aeruginosa  (.33)  klebsiella-pneumoniae  (.3)  diplococcus* 
pneumoniae  (.29)  neisseria-menlngitidis  (.18)  staphylococcus-coag-pos  (.12) 

[RULE  198]. 
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(Continuing  our  discussion  of  the  therapeutic  regimen  of  J.Smith  [RULE092].) 
At  this  point,  MYCIN  would  determine  the  best  therapy  recommendation  from 
among  the  drugs  likely  to  be  effective  against  the  significant  organisms. 
(End  of  our  discussion  of  RULE092) 


{ Discussion  of  the  main  goal  is  now  complete,  and  the  tutor  continues  to  follow  the 
procedure  for  discussing  a  case.  Comparison  of  the  student’s  choice  of  therapy  to 
MYCIN's  has  been  disabled  (see  [Clancey,  1978] /or  details.) 


This  completes  our  discussion  of  PATIENT-638. 

Saving  student  model...done. 

Closing  your  typescript  file  <CLANCEY>TYPESCRIPT.CLANCEY/26-MAR-79;2 
Shall  I  list  it  on  the  line  printer  (Yes,  No,  or  Oelete  it)? 

**  NO 

Oo  you  have  any  comments? 

-NO 

Thank  you. 


The  Student's  Reaction  to  the  Program 
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3.4  The  Student's  Reaction  to  the  Program 

This  single  trial  of  GUIDON  is  not  intended  to  be  a  controlled  evaluation  of  the 
program.  Nevertheless,  it  is  useful  to  consider  this  student's  reaction  to  the  program, 
particularly  remarks  that  were  surprising,  revealing  tutoring  issues  to  consider  in  later 
versions  of  the  program, 

3.4.1  Definition  of  the  Problem 

The  first  problem  is  that  the  student  was  unsure  of  his  responsibility  for  the 
patient.  Was  he  supposed  to  play  the  part  of  a  "resident  physician,"  reviewing  a 
problem  that  other  physicians  had  handled  last  week?  Or  was  this  one  of  his  own 
patients  who  might  require  life-saving  treatment  in  the  next  24  hours?  This  student 
felt  that  knowledge  of  the  setting  was  important  for  "getting  the  proper  frame  of 
mind.”  However,  it  Is  not  clear  whether  this  setting  was  a  necessary  condition  for 
remembering  how  to  solve  the  esse  or  whether  he  would  have  felt  simply  more  at  ease 
if  the  case  were  placed  in  a  familiar  context.1 

As  the  student  received  information  about  the  patient,  he  felt  uncertain  about 
time  relationships.  For  example,  was  the  patient  hospitalized  because  he  was 
burned?  MYCIN  is  particularly  weak  In  this  respect,  for  Its  parameters  are  more  like 
isolated  propositions  than  like  facts  that  form  a  story. 

Perhaps  the  most  difficult  problem  that  the  student  faced  was  In  understanding 


1  Realism  is  frequently  cited  to  be  an  important  design  criterion  for  medical  CAI 
programs  [Feurceig,  at  al„  1  W64][Harle«a,  at  el.,  1871]. 
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MYCIN's  terms.  He  asked,  "How  could  a  current  drug  have  been  discontinued? "  "How 
could  there  be  'organisms  from  a  pending  culture'?"  "How  does  the  'type  of  the 
infection'  differ  from  the  'name  of  the  Infection?"  "Is  a  soft  tissue  infection  the  same 
as  a  wound  infection?"  These  are  all  good  questions  that  required  some  explanation 
before  the  student  could  proceed. 

3.4.2  Strategy 

It  was  a  surprise  to  see  the  student  make  an  hypothesis  on  the  basis  of  the 
Initial  case  data  alone.  He  knew  that  there  was  much  more  information  to  consider, 
but  he  wanted  to  see  how  GUIDON  would  react  to  his  first  thoughts.  Indeed,  the 
student  followed  the  basic  approach  of  using  what  he  knew  as  a  basis  for  gathering 
more  information,  rather  than  exploring  what  MYCIN  knew  about  the  problem.  More 
importantly,  he  did  not  even  fully  read  summaries  of  MYCIN's  evidence  for  an 
hypothesis.  Ail  of  this  changed  when  the  student  said  that  he  exhausted  his 
knowledge  of  how  to  proceed.  As  we  might  expect,  he  then  began  to  request  help  or 
a  quiz.  It  was  at  this  point  that  he  took  up  the  hardcopy  handbook  and  went  down  the 
list,  using  it  to  Jog  his  memory  about  case  data  to  request. 

Just  as  the  student's  cognitive  style  shifted  (from  using  what  he  knew  to  relying 
on  the  program's  suggestions)  as  he  exhausted  his  expertise,  we  found  that  his 
problem  solving  strategy  varied  at  the  same  time  from  attempting  to  confirm  an 
hypothesis,  to  exhaustively  collecting  all  available  data  for  a  goal.  As  long  as  data 
suggested  plausible  hypotheses,  he  pursued  more  data  that  would  build  a  case.  For 
example  to  confirm  Neisseria-meningltidis,  first  suggested  by  the  age  of  the  patient, 
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he  asked  if  the  patient  had  a  rash,  it  was  surprising  how  quickly  he  felt  comfortable 
with  an  hypothesis:  he  did  not  ask  about  the  white  blood  count  in  the  CSP  because  he 
assumed  it  was  high  on  the  basis  of  what  he  already  knew.1 

When  the  student  could  no  longer  think  of  factors  that  would  confirm  his 
hypothesis,  he  tried  to  recall  factors  for  other  hypotheses  and  asked  about  them 
(page  54).  He  characterized  his  approach  as  "jumping  around,  scrambling,  knowledge- 
limited."  Next,  he  used  MYCIN's  lists  of  factors  (and  the  handbook)  to  remind  him  of 
other  relevant  questions  (page  67).  At  this  point  he  was  no  longer  working  on  single 
hypotheses,  though  he  restricted  his  requests  to  data  that  he  knew  how  to  use.  In 
the  final  stage  the  student  was  exploring  MYCIN  relations  that  were  new  to  him. 

3.4.3  Interaction  with  the  Program 

It  is  not  surprising  that  GUIDON'S  initial  orientation  to  the  problem  is  too  long. 
The  student  said  that  he  was  overwhelmed  by  the  goal  rule,  sketch  of  the  subgoal 
tree,  and  orientation  for  determining  the  organisms  that  therapy  should  cover  for.  He 
said  that  he  would  have  preferred  to  have  four  or  five  sentences  to  get  him  started 
on  the  problem.  A  simple  statement  like,  "Your  goal  is  to  ...."  would  have  been  useful. 

It  is  significant  that  the  student  used  the  ORIENT  and  QUIZ  options  immediately. 
He  did  not  know  what  to  do,  and  hoped  that  the  program  would  tip  its  hand  a  bit. 
Unfortunately  the  ORIENT  option  was  not  Intended  to  provide  the  crisp  summary  of 
what  had  occurred  and  was  about  to  take  place  thet  the  student  desired.  Indeed,  at 
first  the  student  did  not  know  what  a  "topic"  was. 

1  Paradoxically,  GUIDON  concluded  thet  he  left  out  the  WBC  in  the  CSF  because 
It  was  too  advanced  for  him  to  consider  (page  67). 
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As  has  been  mentioned,  the  student  did  not  read  everything  that  GUIDON  printed. 
Sometimes  he  became  weary  of  reading,  and  sometimes  he  skipped  over  statements  in 
his  exuberance  to  get  on  with  the  problem.  Consequently,  he  missed  the  statement 
about  "partially  treated  meningitis"  (page  66).  He  missed  the  definition  of  "the  type 
of  the  Infection"  (page  53)  and  later  asked,  "What  kinds  of  answers  does  the  program 
want?" 

The  persistence  of  the  student's  interest  was  especially  surprising.  During  the 
response  to  the  student's  hypothesis  about  the  type  of  the  infection,  the  program 
went  on  to  quiz  him  about  hypotheses  he  did  not  mention,  while  the  student  still 
wanted  to  talk  about  the  evidence  for  a  bacterial  Infection.  "What  was  the  CSF 
glucose  value?"  he  asked.  The  program  takes  up  new  subjects  more  quickly  than  a 
human  usually  does.  However,  it  should  be  noted  that  the  discussion  to  wrap-up 
consideration  of  a  rule  (page  76),  including  mention  of  related  rules  (page  77),  is 
designed  to  Imitate  the  typical  way  humans  slow  down  before  taking  up  other  major 
subproblems.  Rapid  fire  questioning,  as  takes  place  in  response  to  a  student's 
hypothesis,  probably  does  not  accommodate  a  typical  student's  pace  for  dig*>«ting 
new  information. 

The  student  often  reread  earlier  portions  of  the  dialogue,  both  to  keep  track  of 
what  had  been  determined,  and  to  follow-up  on  remarks  made  by  the  program.  This 
suggests  that  modeling  strategies  might  use  the  remark  indices  that  the  student 
mentions  as  clues  about  what  he  has  read  or  gone  back  to  reread,  contributing  to  the 
model  of  his  thinking  process  (see  Section  8.1). 
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Finally,  when  asked  to  summarize  his  reaction  to  the  program,  the  student  raised 
a  provocative  question.  He  felt  that  GUIDON  was  more  sophisticated  than  other 

tutorial  programs  that  he  had  used.1  But,  in  contrast  with  simple  CAI  programs  with 
limited  options  in  which  "you  are  always  In  control,"  he  began  to  lose  confidence  in  his 
model  of  the  program.  The  complexity  of  the  dialogue  (quizzing,  stating  results, 
interrupting  to  present  rules,  summarizing  evidence)  gave  the  tutor  an  air  of 
unpredictability.  Because  the  student  did  not  feel  that  he  had  a  good  model  of  the 
program's  methods,  he  wondered  if  the  program  really  had  a  good  model  of  him.  It  Is 
possible  that  this  student  would  feel  more  comfortable  (and  more  in  control)  in  later 
sessions,  after  becoming  more  familiar  with  the  options  available  and  understanding 
the  typical  ways  in  which  GUIDON  takes  initiative. 


1  The  student  had  previously  used  the  CASE  system  [Harless,  at  al.,  1971]  and 
Massachusetts  General  Hospital  programs. 
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The  instructional  paradigm  of  the  case  method  involves  presenting  particular 
problems  to  a  student,  and  providing  a  supportive  environment  to  help  him  solve  the 
problem.  The  major  objective  of  the  GUIDON  project  has  been  to  construct  a  case- 
method  tutor  that  uses  (a)  knowledge  of  the  domain  and  (b)  knowledge  of  tutoring 
strategies,  while  keeping  the  two  distinct.  This  chapter  provides  an  overview  of  the 
knowledge  representations  of  both  domain  expertise  (for  solving  the  problems 
presented  to  the  student)  and  tutoring  expertise  (for  carrying  on  a  dialogue  with  the 
student  as  he  solves  a  problem).  We  emphasize  here  two  crucial  aspects  of  this 
separation  of  domain  knowledge  from  the  tutorial  program:  1)  How  Is  the  problem 
solution  communicated  to  the  tutor?  and  2)  What  knowledge  about  tutorial  dialogues 
will  the  program  need?  Chapter  5  goes  on  to  consider  what  domain  knowledge,  in 
addition  to  that  which  MYCIN  uses  to  solve  the  problem,  is  useful  for  tutoring. 

GUIDON  uses  a  MYCIN  consultation  program  as  a  source  of  problem  solving 
expertise.  Below  we  describe  the  MYCIN  formalism  (Section  4.2),  and  characterize 
its  assets  and  limitations  (Section  4.3).  Here  we  focus  on  the  use  of  the 
infectious  diseases  knowledge  base;  Chapter  B  reports  on  the  implementation  of 
GUIDON  in  two  other  domains. 

4.1  Components  of  the  GUIDON  System 

Figure  4.1  shows  the  configuration  of  the  GUIDON  system.  The  most  important 
feature  Is  that  the  the  domain  knowledge  Is  distinct  from  the  teaching  expertise.  The 
"expert"  (MYCIN)  consists  of  an  interpreter  for  applying  the  domain  knowledge  to 
particular  problems,  producing  a  store  of  dynamic  knowledge  which  Is  accessible  to 


Component j  of  the  GUIDON  System 


83 


the  tutor.  The  tutor  can  access  all  of  the  knowledge  base  as  well,  and  uses  It  to 
discuss  particular  aspects  of  the  problem  solution  with  a  student.  Notice  that 
additional  domain  knowledge  has  been  provided  for  use  In  tutoring.  (This  is  described 
in  Chapter  5.)  The  main  component  of  the  tutor  module  is  a  collection  of  tutoring 
rules,  hereafter  t-rulas,  for  carrying  on  a  dialogue,  including  methods  for  providing 
assistance,  maintaining  a  student  knowledge  model,  responding  to  his  Initiative,  and  so 
on.  Beginning  with  the  expert  system,  components  of  this  framework  are  described 
further  below. 


USER 


STUDENT 


Figure  4.1.  Modules  for  GUIDON)  A  Domain  Independent  Tutorial  System 


("K"  =  Knowledge) 
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4.2  The  EMYCIN  Formalism 

The  MYCIN  knowledge  base  contains  approximately  450  production  rules  [Davis 
&  King,  1977].  Each  rule  consists  of  a  set  of  preconditions  (called  the  premise) 
which,  if  true,  justifies  the  conclusion  made  in  the  action  part  of  the  rule.  An  example  is 
shown  below.  We  call  each  precondition  a  subgoal.  If  all  of  the  subgoals  in  the 
premise  can  be  achieved  (shown  to  be  true),  then  a  conclusion  can  be  made  about  the 
goal  in  the  action.  A  classification  of  the  predicate  functions  used  in  MYCIN  rules  is 
given  in  Appendix  A. 

IF  1 )  the  site  of  the  culture  is  one  of:  those  sites  that  are  normally 
sterile,  and  2)  the  gram  stain  of  the  organism  is  gram  negative, 

THEN  there  is  strongly  suggestive  evidence  (.8)  that  there  is 
significant  disease  associated  with  this  occurrence  of  the  organism. 

PREMISE:  (SAND  (SAME  CNTXT  SITE  (LISTOF  STERILESITES) ) 

(SAME  CNTXT  GRAM  GRAMNEG) ) 

ACTION:  (CONCLUDE  CNTXT  SIGNIFICANCE  YES  TALLY  808) 

Figure  4.2.  Sample  MYCIN  Rule  in  English  and  LISP  Form 

In  addition  to  the  rules,  there  are  several  hundred  facts  and  relations  stored  in 
tables,  which  are  referenced  by  the  rules,  e.g.,  the  list  of  body  sites  that  are  normally 
sterile  is  referenced  by  the  rule  shown  above. 

It  is  possible  to  remove  the  medical  knowledge  from  MYCIN  program  and 
substitute  a  set  of  rules  about  a  new  domain.  The  consultation  system  will  provide 
advice  based  on  the  new  rules  and  the  explanation  system  will  explain  the  reasoning 
behind  the  advice.  The  domain-independent  package  consisting  of  the  rule 
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interpreter,  explanation,  and  knowledge  acquisition  modules  is  called  EMYCIN 
("essential  MYCIN").1  One  EMYCIN  program,  called  SACON  [Bennett,  et  al.,  1978],2 
provides  advice  on  structural  analysis  problems,  such  as  what  stress  and  deflection 
behaviors  should  be  observed  in  an  airplane  wing  mado  of  a  certain  material  and 
subject  to  certain  loading  conditions.  In  addition,  the  EMYCIN  system  has  been 
applied  to  two  very  different  medical  domains:  interpretation  of  pulmonary  function 
tests  (PUFF)  [Kunz,  et  al„  1978]  and  drug  therapy  recommendations  for  psychiatric 
patients  (HEADMED)  [Heiser,  Brooks,  &  Ballard,  1978]. 

4.3  Applicability  of  EMYCIN  Systems  for  Tutorials 

In  this  section  we  make  observations  relevant  to  converting  MYCIN  to  a  tutorial 
program.  Here  assets  and  properties  of  the  EMYCIN  representation  and  the  MYCIN 
infectious  disease  knowledge  base  are  considered;  Chapter  5  considers  forms  of 
domain  knowledge  that  are  distinct  from  the  performance  rules  which  MYCIN  uses  for 
solving  problems. 

4.3.1  Assets  of  the  Formalism 

In  what  ways  are  MYCIN-like  systems  particularly  well*sulted  for  use  in  tutoring? 

I  First,  the  existence  of  representational  meta-knowledge  gives  the  tutorial 
program  the  capability  to  take  apart  the  rules  and  use  them  to  form  quizzes,  to  guide 


1  EMYCIN  Is  being  developed  by  William  van  Melle  and  A.  Carlisle  Scott  [van 
Melle,  1979], 

2  Collaborative  project  with  the  MARC  Corporation,  Inc. 
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the  dialogue,  to  summarize  evidence,  to  model  the  student,  and  so  on.3  Flexible 
access  to  the  knowledge  store  makes  it  possible  to  provide  a  variety  of  interactive 
aids.  For  example,  the  student  can  use  the  program  to  evaluate  his  problem-solving 
ideas,  or  use  the  program  to  as  an  information  retrieval  system  see  directly  what  the 
program  knows.  In  addition,  this  flexibility  permits  us  to  write  a  variety  of  tutoring 
rules  that  select  and  present  teaching  material  in  multiple  ways  (including  quizzes, 
direct  statements,  and  hints)— an  important  consideration  if  we  are  to  use  the 

MYCIN/GUIDON  system  for  experimenting  with  different  teaching  strategies.1 

■  Second,  the  rules  themselves  are  well-defined  problem-solving  methods:  the 
premise  states  explicitly  when  the  method  can  be  applied,  and  the  action  states  the 
evidence  that  should  be  added  to  the  knowledge  store.  It  is  convenient  to  think  of 
the  list  of  rules  for  determining  a  MYCIN  parameter  as  a  bag  of  techniques  for 
achieving  a  goal.  Rule  preconditions  can  be  called  "factors,"  "subgoals,”  or 
"deltacts”  [Brown,  Collins,  &  Harris,  1977],  as  we  find  convenient.  Rule  actions  are 
goals,  steps  in  diagnostic  reasoning,  or  alternative  hypotheses,  again  according  to  our 
perspective  at  the  moment.  The  subgoal  structure  of  the  rule  set  was  designed  to 
break  down  the  problem  In  a  conventional  way  that  would  facilitate  explanation,  and 


3  Function  templates  are  the  most  important  kind  of  representational  meta¬ 
knowledge  in  MYCIN  systems.  Templates  "indicate  the  order  and  generic  type  of  the 
arguments  in  a  typical  call  to  the  function”  [Davis  &  Buchanan,  1977].  The  template 
for  the  function  SAME  (see  Figure  4.2)  is  "(SAME  CNTXT  PARM  VALU)."  Using  this 
template  to  "read  the  rules,"  GUIDON  can  extract  "(IISTOF  STERILESITES)"  from  the 
rule  above  to  determine  the  value  for  "the  site  of  the  culture”  that  will  satisfy  the 
precondition.  Templates  are  also  used  to  reconfigure  the  output  of  the  consultation 
(Section  4.4.1  and  Appendix  A). 

1  Note  that  the  ability  to  present  domain  knowledge  in  multiple  ways  is 
characteristic  of  a  generative  tutor.  Problem-solving  knowledge  is  not  "compiled  in”; 
given  appropriate  tutoring  methods,  the  program  can  reason  about  what  it  knows, 
adapting  subject  material  to  the  needs  of  a  particular  dialogue  with  a  student. 
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this  turns  out  to  be  a  useful  framework  tor  organizing  the  topic  structure  of  a 
dialogue  about  a  particular  problem. 

Considering  the  rules  as  methods  or  skills,  we  can  superimpose  taxonomies 
that  relate  them  according  to  difficulty  and  typical  usefulness,  or  group  them 
structurally  by  syntax,  analogy,  prerequisite,  or  specialization/generalization  links. 
Thus,  there  is  a  distinction  between  the  "raw"  expertise  that  links  situations  to 
actions  and  more  theoretical  concepts  that  show  how  this  knowledge  is  interrelated. 
For  the  purpose  of  improving  the  program  or  experimentation,  new  organizations  can 
be  introduced  independently  of  one  another,  without  modifying  the  rules  themselves. 

The  (unordered)  bags  of  rules  require  no  particular  control  structure  for 
applying  them  to  solve  problems.  In  consultations  we  can  use  backward  chaining  (to 
restrict  the  questions  asked  to  just  those  needed  to  apply  rules  In  a  particular  case), 
while  in  a  tutorial  we  can  run  the  rules  forwards  to  model  the  expert  and  student,  or 
even  order  them  according  to  a  strategy  for  hypothesis  revision  (Section 
6. 2. 3.2).  Using  forward-directed  inferencing  it  is  possible  to  tutor  the  student 
about  redundancy  and  relevancy  of  the  information  he  gathers.  When  he  states 
and/or  supports  an  hypothesis  for  a  subproblem,  it  Is  simple  to  see  If  the  evidence  is 
consistent  with  his  hypothesis.  Thus,  some  of  the  important  distinctions  between  the 
MYCIN  consultation  system  and  systems  like  INTERNIST  begin  to  blur  with  the  addition 
of  control  schemes  and  meta-knowledge  for  tutoring.  In  addition,  the  rules  represent 
uncertain  judgments,  so  it  is  possible  to  discuss  relative  pros  and  cons  for  each 


decision. 
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I  Third,  development  of  MYCIN's  question-answer  (QA)  facility  was  useful 
ground  work  for  a  tutoring  system.  For  example,  for  the  QA  system  to  work  properly, 
it  was  necessary  to  keep  the  rules  "clean,"  by  keeping  the  meta-knowledge  that 
describes  them  up-to-date,  and  never  changing  the  formalism  in  a  way  that  would 

prevent  a  program  from  reading  the  rules.  Moreover,  design  of  the  QA  system  1  made 
it  necessary  to  work  out  in  full  detail  all  kinds  of  program  outcomes,  for  example,  all  of 
the  reasons  why  a  factor  would  not  be  pursued  by  MYCIN  during  consultation. 

I  Fourth,  the  development  of  domain-specific  meta-knowledge  [Davis  S» 
Buchanan,  1977]  in  the  form  of  rule  models  and  metarules  (Section  5.1  4  and 
Section  6.3.2)  was  the  seed  for  all  of  our  subsequent  work  on  meta-knowledge 
(Chapter  6).  Preliminary  studies  using  this  meta-knowledge  for  knowledge 
acquisition  and  explanation  led  directly  to  its  use  in  GUIDON  for  orienting  the  student, 
summarizing  evidence,  and  so  on.  Moreover,  continuing  research  with  GUIDON  is  very 
much  directed  by  the  now  evident  parallel  between  the  knowledge  acquisition  program 
(transferring  expertise  from  human  to  program)  and  tutorial  program  (transferring 
expertise  from  program  to  human  student). 


I  Fifth,  the  production  rule  formalism  used  by  MYCIN  is  widely  applicable  to 
tasks  other  than  medicine,  although  it  Is  by  no  means  a  "universal"  language.2  The 


1  The  initial  system  was  written  by  Ted  Shortliffe.  Later  improvements  were 
made  by  A.  Carlisle  Scott,  the  author,  and  William  van  Melle  [Davis,  1976]  [Scott,  et 
al„  1977]. 

2  Production  rules  have  worked  well  in  several  domains  [Feigenbaum,  1977]. 
For  example,  PROSPECTOR  [Hart,  1976],  the  SRI  International  program,  was  designed 
originally  as  a  MYCIN-like  system  for  consulting  with  geologists  about  mineral 
exploration  sites.  SU/X  is  a  signal  processing  program  for  military  applications;  it  also 
makes  use  of  a  production  rule  formalism  [NIi  &  Feigenbaum,  1977].  However,  other 
approaches  for  building  knowledge-based  systems  are  possible.  For  example,  Pople 
successfully  uses  frame-like  disease  hypotheses  [Pople,  1977],  and  Weiss  and 
Kulikowski  use  a  causal-assoclational  network  [Weiss,  Kulikowski,  8  Sefir,  1977]. 
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tutoring  system  will  work  with  cases  and  rules  in  any  EMYCIN  system,  assuming  some 
parallels  between  the  structure  of  the  knowledge  in  the  new  domain  and  the  structure 
of  the  existing  medical  knowledge  (see  Chapter  9  for  discussion). 


4.3.2  Limitations  of  the  Original  Explanation  Capability 


The  production  rule  formalism  Is  the  key  to  MYCIN's  explanation  capabilities: 
steps  in  the  problem  solution  are  associated  with  specific  rules  whose  applicability  is 

well-defined.  A  typical  QA  interaction  is  shown  below.1 

**  DO  YOU  EVER  USE  THE  AGE  OF  THE  PATIENT  TO  CONCLUDE  DIPLOCOCCUS? 

The  following  rules  use: 

the  age  of  the  patient 
to  conclude  about: 

the  likelihood  that  Dlplococcus-pneumonlae  is  one  of  the  organisms 
(other  than  those  seen  on  cultures  or  smears)  which  might  be 
causing  the  Infection 

535,  534.  554,  587,  341 

Which  of  the  rules  do  you  wish  to  see? 

••  NONE 


Figure  4.3.  General  Question  for  MYCIN's  QA  System 


Experience  with  MYCIN's  QA  system  leads  to  the  conclusion  that  reading  back  a 
trace  of  problem-solving  steps  (the  rules)  will  not  be  sufficient  for  a  tutorial  system. 
Indeed,  this  is  easily  observed  by  any  naive  user  who  tries  to  unravel  MYCIN's 
reasoning.2  Some  of  the  lessons  we  have  learned  about  explanation  are  summarized 
below. 


1  The  system  simply  does  one-level  Indexing  by  rule  preconditions  and 
conclusions;  no  chaining  of  inferences  is  considered. 

2  It  is  worth  noting  that  Swartout,  using  an  entirely  different  representation  of 
problem-solving  knowledge  (OWL  procedures),  has  encountered  some  of  the  same 
difficulties  in  providing  explanations  for  the  Digitalis  therapy  advisor  [Swartout, 
1077]. 
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1 )  The  explanation  system  should  summarize  conclusions  by 
hierarchically  describing  key  reasoning  steps,  rather  than 
providing  a  list  of  all  of  the  rules  that  were  applied.  The  real 
problem  in  explanation  is  to  abstract  from  performance- -not  merely 
to  say  what  was  done,  but  why  it  was  done  and  what  alternatives 
were  considered. 


2)  Strictly  speaking,  a  program  that  lacks  a  user  model  cannot 
be  said  to  have  an  "explanation"  capability.  There  is  an  important 
difference  between  passively  retrieving  information  and  restating  it 
for  a  particular  listener. 


3)  A  naive  user  doesn't  know  where  to  begin  asking  questions. 
It  is  important  that  an  explanation  program  convey  the  general 
structure  of  the  solution  process  so  that  a  user  has  a  handle  for 
pursuing  details.  The  vocabulary  must  be  shared. 


4)  Problem-solving  steps  may  bear  only  indirect  relation  to 
underlying  mechanism  models  for  the  domain.  Expertise  for 
efficiently  cracking  a  problem  is  in  some  cases  orthogonal  to  the 
general  principles  that  are  often  taught  to  a  beginner.  (See  Chapter 
6  for  discussion.) 


These  are  difficult  problems,  though  some  progress  has  been  made  with  all  of 
them  in  GUIDON.  The  main  question  asked  by  Chapter  6  is:  if  teaching  involves  more 
than  simply  reciting  expert  performance  rules,  what  else  does  the  tutor  need  to  say? 


4.3.3  Other  Rule  Set  Properties  Important  for  Tutoring 

To  some  the  degree,  GUIDON'S  design  and  ultimate  capability  to  teach  is  a 
r  aflection  on  the  particular  rule  set  available.  Some  of  the  general  properties  of  the 


infectious  disease  rule  set  are  considered  below. 
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Completeness 


The  success  and  usefulness  of  GUIDON'S  tutorial  ability  depends  to  some  extent 
on  the  universality  of  the  rule  set,  i.e.,  how  the  subject  matter  has  been  structured  by 
the  rule  authors.  The  domain  rules  were  designed  to  be  comprehensible  to  potential 
users  of  the  consultation  system  who  are  not  experts,  but  it  is  inevitable  that  some  of 
the  concepts  represented  in  the  rules  are  artifacts  of  the  particular  representation 
and  the  requirements  of  making  everything  explicit.  Furthermore,  GUIDON  acts  as  if 
the  domain  of  discourse  is  closed  (i.e.  limited  to  the  expert  rule  base),  when  in  fact  it 
is  not:  the  student  may  want  to  discuss  controversial  diagnostic  procedures  not 
known  to  MYCIN. 


Validity 


Two  formal  evaluations  of  MYCIN's  performance  have  demonstrated  that  MYCIN's 
competence  in  selecting  antimicrobial  therapy  for  meningitis  and  for  bacteremia  is 
comparable  to  that  of  the  infectious  disease  faculty  at  Stanford  University  School  of 
Medicine  (where  MYCIN  was  developed)  [Yu,  et  el.,  1978].  From  this  we  conclude 
that  MYCIN's  rules  capture  a  significant  part  of  the  knowledge  necessary  for 
demonstrably  high  performance  in  this  domain.  The  fact  that  the  formalism  of 
production  rules  has  been  exploited  in  MYCIN  to  create  a  rich,  high  performance 
knowledge  base  for  solving  difficult,  real  world  problems  Is  an  important  starting  point 
for  demonstrating  the  advantages  of  using  this  representation  of  domain  knowledge 
for  tutoring. 


Size 


The  effectiveness  of  GUIDON'S  current  dialogue  strategy  of  stacking  topics 
partially  depends  of  the  depth  of  the  AND/OR  solution  tree.  In  MYCIN  it  Is  usually  only 
two  levels  below  the  top  goal  of  determining  therapy  for  the  patient.  The  other 
EMYCIN  systems  generally  share  this  property.  (See  Chapter  0  for  further 
discussion.) 


On  the  other  hand,  the  very  fact  that  MYCIN  has  hundreds  of  rules  has  made  it 
important  to  develop  tutoring  strategies  for  carrying  on  the  dialogue  as  economically 
as  possible.  There  is  no  time  to  explicitly  discuss  with  the  student  each  inference 
that  MYCIN  attempted  during  the  corresponding  consultation.  Section  6.2.1 
deals  with  this  problem  in  detail. 
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Single  Form  of  Expertise 


Multiple  forms  of  expertise  are  sometimes  important  for  tutoring.  For  example, 
when  mechanistic  reasoning  is  involved,  both  qualitative  and  quantitative 
representations  may  be  useful  to  solve  the  problem.  DeKleer  has  found  that 
strategies  for  debugging  an  electronic  circuit  are  "radically  different"  depending  on 
whether  one  does  local  mathematical  analysis  (using  Kirchoff's  laws)  or  uses  a  higher 
level,  functional  analysis  of  components  [Brown  &  Burton,  187S][Brown,  Rubinstein, 
&  Burton,  1976]  [de  Kleer,  1979].  One  might  argue  that  a  tutor  for  the  electronics 
domain  should  be  ready  to  recognize  and  generate  arguments  on  both  of  these  levels. 
(See  [Carr  8>  Goldstein,  1977]  for  related  discussion.) 


The  MYCIN  program  demonstrates  that  an  adequate  set  of  rules  for  reasoning  in 
infectious  disease  problem-solving  can  be  based  on  judgments  about  empirical 
information,  rather  than  arguments  based  on  causal  mechanisms  [Weiss,  Kuiikowski, 
&  Safir,  1977].  MYCIN's  judgments  are  "cookbook"  responses  that  address  the  data 
directly,  as  opposed  to  attempting  to  explain  it  in  terms  of  physiological  mechanisms. 
Even  though  MYCIN's  domain  apparently  makes  it  possible  for  cases  to  be  solved 
without  recourse  to  the  level  of  physiological  mechanisms,  a  student  may  find  It  useful 
to  know  this  "support"  knowledge  that  lies  behind  the  rules  (see  Section  6.2). 


Weak  Model  of  Inquiry 


There  is  no  strong  model  for  ordering  the  collection  of  evidence  in  MYCIN  1 
Medical  problem  solving  has  recourse  to  strategies  for  building  cases,  but  there  are  no 
hard  and  fast  rules  for  evaluating  a  student's  approach.  While  there  are  some 
conventions  that  ensure  that  all  routine  data  are  collected,  physicians  have  no  agreed 
upon  basis  for  numerically  optimizing  the  decision  of  what  to  do  next.  (See,  for 
example,  [Sprosty,  1963].)  The  uncertainty  of  procedures  for  reasoning  in  this 
domain  suggests  that  a  tutor  will  want  to  accommodate  different  problem-solving 
approaches. 


1  In  Goldstein's  WUMPUS  program,  for  example,  it  is  possible  to  rank  each  legal 
move  (analogous  to  seeking  case  data  in  MYCIN)  and  so  rate  the  student  according  to 
rejected  Inferior  moves  and  miu«d  superior  moves.  The  same  analysis  Is  possible  In  Burton 
and  Brown's  version  of  the  WEST  program. 


GUIDON:  Conversion  of  tht  Consultation  System  1 03 

4.4  GUIDON:  Convarsion  of  tha  Consultation  System 

4.4,1  The  Consultation  Record  and  Its  Reconfiguration 

MYCIN's  production  rules  ere  applied  by  an  Interpreter  using  a  backward  chaining 
control  scheme  Backward  chaining  can  be  described  as  follows: 

If  a  condition  in  the  IF  (premise)  part  of  a  rule  Is 
decidable  from  the  database,  the  value  is  noted  and  used 
directly;  if  the  condition  can  be  asserted  by  the  THEN  (action) 
part  of  some  other  rules,  they  are  applied;  otherwise,  MYCIN 
asks  the  user,  (after  [Szolovits  <■  Pauker,  1978],  p.  136) 

The  output  of  the  consultation  includes  case  data  acquired  from  the  user  and 
conclusiors  made  from  rules  that  were  successfully  applied.  For  GUIDON'S  purposes, 
MYCIN's  compound  predicate  functions  (corresponding  to  AND,  OR,  "there  exists  an  X 
such  that  P,”  "there  is  a  maximum  (minimum)  X  such  that  P,"  and  "there  is  a  subset  of 
L,  such  that  P")  have  been  modified  to  leave  behind  traces  of  their  evaluation  for 
later  analysis.  These  traces  indicate  which  clauses  of  the  compound  predicates  were 
tried,  the  context  of  evaluation,  and  whether  they  succeeded.  As  such,  the  traces 
are  significantly  more  complex  than  those  used  by  the  post-consultation  question- 
answer  program  In  MYCIN.  (The  precise  format  and  recursive  construction  of  the 
traces  is  described  in  Appendix  A.) 

Next,  In  batch  mode,  GUIDON  reconfigures  the  consultation  traces  to  form  a 
complete  AND/OR  tree  representation  of  the  problem  solution.  The  analysis  is  non¬ 
trivial,  for  It  requires  that  the  nesting  of  predicates  be  taken  Into  account.  For 
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example,  if  (NOT  (THEREXISTS  an  X  such  that  P))  is  a  false  rule  precondition,  then  the 
program  must  record  that  every  element  of  X  must  be  known  before  the  rule  can  be 
evaluated.  Moreover,  the  tree  is  complete  in  that  it  explicitly  records  all 
preconditions  of  a  rule  that  fail.1  (This  information  is  crucial  for  applying  the  rules  in  a 
forward  direction  during  the  tutorial  to  model  the  student's  understanding  (see  Section 
4.4.2  below).)  The  resulting  tree  allows  the  tutorial  program  to  efficiently  guide 
the  dialogue  without  continuously  re-analyzing  the  rules  for  relationships  between 
subgoals. 


Figure  4.4  shows  the  basic  outline  of  the  AND/OR  tree.  A  parameter  goal 
(actually  a  parameter/context  pair,  such  as  SITE/CULTURE- 1 )  is  an  OR  node  with 
three  groups  of  rules,  those  that  failed,  succeeded,  and  were  not  used.  These  rules 
are  AND  nodes  whose  preconditions  constitute  subgoals.  Pointers  tie  goals,  rules  and 
subgoals  in  all  directions. 


Pointers  Linked  Nodes 

SG  goal-subgoal 

UPDATE  goal-rules 

CONCLUSIONS  rule-hypothesis  values  of  goal 

USE  rule-subgoals 

Figure  4.4.  General  Form  of  the  AND/OR  Tree 


1  But  see  Appendix  A  for  a  discussion  of  the  logical  problems  involved,  and 
insufficiencies  of  the  current  approach. 
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A  typical  OR  node  is  shown  in  Figure  4.5.  Observe,  for  example,  that  the 
parameter  COVERFOR  Is  used  successfully  in  rules  198  and  92,  and  rules  578  and 
1 60  are  two  rules  that  determine  COVERFOR-  COVERFOR  Is  a  subgoal  for  REGIMEN  and 
TREATINF  is  one  of  its  subgoals.  The  RACE  of  the  patient  appears  in  a  rule  that  failed 
to  make  a  conclusion  about  COVERFOR  in  this  case.  (Note  that  only  parameter  and  rule 
names  are  shown  here.  In  the  tree,  a  context  Is  always  included,  e.g.,  ORGANISM-2.) 


USE-SUCCI:  (RULE  198  RULE892) 


COVERFOR- 


UPDATE-SUCC! : 

(RULES78  RULE  168. . . ) 


UPDATE-FAIL! : 

(RULE002  RULE869. . . ) 


CONSEQ^-SG ! :  UNUSED-SG ! : 

(TREATINF  SPECSTAIN  BURNEO...)  (EXAMSTAIN  TBRISK. . . ) 


FA^L-SG]  : 


COMPROMISED 
RACE...) 


KEY:  USE-SUCC!  =  subgoal  In  rules  that  succeed 
UPDATE-SUCC!  *  rules  used  to  determine  goal 
UPDATE-FAIL!  *  rules  that  failed  to  apply 
CONSEQ-SG!  =  subgoals  In  rules  that  succeed 

UNUSED-SG!  =  subgoals  that  appear  In  rules  for  determining  the 
parameter,  but  don’t  need  to  be  known  (e.g., 
true  preconditions  of  rules  that  fall) 

FAIL-SG!  =  subgoals  that  cause  rules  to  fall 


Figure  4.6.  OR  node  in  the  reconfigured  tree 
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A  portion  of  the  consultation  record  that  was  reconfigured  to  form  this  tree  is 
shown  in  Figure  4.6.  The  components  of  the  record  and  logical  reconfiguration  of  the 
rule  (SUCC-PARMS!)  are  described  In  the  key. 


APPLIED-TRACE: 


[(CULTURE-1  NIL 
(2  SUCCEED  SOR  2)] 


'S\tULE578/^/ 


CONCLUSIONS: 


[(DIPLOCOCCUS  737) 
(NEISSERIA  727)] 


SUCC-PARHS!:  (AND  TREATINF  (AND  SPECSTAIN) 
TYPE  BURNED) 


KEY:  APPLIED-TRACE  =  consultation  record,  Indicates  that  clause  2 
of  the  internal  OR  succeeded. 

CONCLUSIONS  *  consultation  record,  indicates  two  organisms  and 
their  certainty  factors. 

SUCC-PARHS!  =  logical  reconfiguration  based  on  the  rule's 

premise  and  applied-trace;  indicates  that  when 
the  parameters  TREATINF,  SPECSTAIN,  TYPE,  and 
BURNED  are  known,  this  rule  should  be  marked  as 
successful  and  Its  conclusions  applied  to  other 
rules. 


Figure  4.6.  ANO  node  in  the  reconfigured  tree 


To  illustrate  some  uses  of  this  tree,  Figure  4.7  shows  one  portion  without  labels 
for  the  pointers.  "COVERFOR"  signifies  the  goal  to  determine  which  organisms  should 
be  considered  for  treatment;  rule  578  concludes  about  this  goal;  "TYPE"  Is  a  subgoal 


of  this  rule. 
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COVERFOR 


RULE  578 


<other  rules> 


TREATINF  TYPE  SPECSTAIN  BURNED  ABNORMAL-X-RAY 

/l\ 

<rules> 


CSFGLUCOSE 

Figure  4.7.  A  portion  of  the  AND/OR  tree  of  goals  and  rules 


Tutorial  rules  make  frequent  reference  to  this  data  structure  in  order  to  guide 
the  dialogue.  For  example,  the  response  to  the  request  for  help  shown  in  the  protocol 
(page  68,  "look  for  diplo")  is  based  first  of  all  on  the  rules  that  were  used  by  the 
expert  program  to  determine  the  current  goal.  Similarly,  the  t-rules  for  supplying  case 
data  requested  by  the  student  check  to  see  if  the  expert  program  asked  for  the 
same  information  during  the  corresponding  consultation  (e.g.,  the  "csf  glucose"  on 
page  64). 


Figure  4.7  is  useful  for  illustrating  several  characteristics  of  EMYCIN  production 
rules  that  must  be  considered  during  the  tutorial  dialogue.  First,  we  define  a  factor  of 
a  goal,  G,  to  be  those  subgoals  that  appear  on  the  first  level  In  the  tree  below  G. 
Thus,  TREATINF,  SPECSTAIN,  TYPE,  BURNED,  and  ABNORMAL-X-RAY  are  all  factors  of 
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COVERFOR.1  Sometimes  when  we  look  at  a  particular  rule,  say  rule  678,  we  find  it 
useful  to  distinguish  between  factors  that  are  determined  by  case  data  and  those 
that  are  determined  by  applying  other  rules.  We  will  call  factors  determined  by 
applying  other  rules  subgoals  in  this  context.  Thus,  TYPE  Is  a  subgoal  in  rule  678, 
while  BURNED  is  a  case  data  factor. 

Second,  the  ANO/OR  tree  is  used  by  the  tutor  to  relate  topics  to  one  another  in 
response  to  student  options.  We  see  on  page  64  that,  during  the  discussion  of  the 
type  of  the  infection,  the  student  requested  information  about  the  patient's  chest  x- 
ray.  Since  ABNORMAL-X-RAY  is  not  a  factor  of  TYPE  and  is  not  a  factor  of  any  subgoal 
of  TYPE,  it  is  defined  to  be  irreltvant  to  the  current  goal.  Given  that  GUIDON  is  using  a 
pre-compiled  solution,  it  must  cope  with  student  reasoning  that  is  apparently  not 
connected  to  this  tree.  See  Section  6.2.2.1  and  Section  7.1. 1.2  for 
discussion. 

In  EMYCIN  systems,  a  distinction  is  drawn  between  those  parameters  that  are 
determined  by  trying  rules  before  asking  the  user  for  a  value,  and  those  parameters 
for  which  one  should  ask  first.  The  following  table  shows  the  four  possible  kinds  of 
Parameters: 


1  The  entire  list  is  computed  by  forming  the  union  of  consequent,  antecedent, 
and  failure  subgoals. 
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Are  there  rules? 


Data  usually 
known  by 
the  user 


Requestable 


Can-'t  ask 


Yes  No 


Ask, 

then  try 
rules 

Ask 

only 

Rules, 

then 

ask 

X 

Rules 

only 

X 

Figure  4.8.  Kinds  of  Parameters 


GUIDON  has  to  determine  the  appropriateness  of  asking  for  case  data  at 
particular  times  in  the  problem  solution  by  checking  the  kind  of  parameter  being 
requested  by  the  student  and  examining  the  status  of  the  expert  program's  solution. 
For  example,  on  page  70,  we  see  that  the  student  requested  ALCOHOLIC,  a  parameter 
that  one  should  try  to  deduce  from  rules  before  asking  the  user.  Since  the  expert 
program  had  not  yet  been  able  to  try  these  rules  (it  is  not  yet  known  whether  the 
patient  is  IMMUNOSUPPRESSED),  it  is  premature  for  the  student  to  request  this 
information.  Section  0.2.2. 1  discusses  how  GUIDON  copes  with  this  situation 
and  others  similar  to  it. 

Finally,  we  will  frequently  refer  to  an  hypothesis  for  a  goal.  For  in  /false 
parameters,  the  value  of  the  hypothesis  can  be  either  "yes"  or  "no,"  as  in  "I  think  that 
the  patient  has  had  a  splenectomy,"  For  parameters  like  "the  type  of  the  infection," 
an  hypothesis  must  be  a  single  value  from  finite  list  of  possibilities;  so  we  might  say 
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that  "the  student  is  considering  that  the  type  of  the  infection  is  bacterial."  For  a  third 
kind  of  parameter,  it  is  possible  for  more  than  one  hypothesis  to  be  correct.  "The 
organisms  that  are  causing  the  infection"  is  an  example--more  than  one  organism 
might  actually  be  present.  These  distinctions  play  a  role  In  hypothesis  evaluation  and 
formation  of  questions  from  the  rules.  (On  the  most  basic  implementation  level,  they 
are  important  considerations  when  constructing  tutorial  remarks  from  text  strings.) 
Section  6.2.2  gives  many  other  examples  of  the  use  of  domain  logic  in  the 
tutorial  dialogue. 

4.4.2  The  Communication  Model 

During  the  tutorial  session,  GUIDON  maintains  a  record  of  the  communication 
between  tutor  and  student.  The  most  important  component  of  the  record  is  the 
student  knowledge  or  overlay  model.  This  model  indicates  for  every  rule:  a)  the  belief 
that  the  student  knows  the  rule,  and  b)  the  belief  that  he  has  applied  it  in  his  solution 
of  the  case  under  discussion.  For  each  goal,  the  model  indicates  the  combined 
conclusion  that  the  student  can  draw,  given  the  rules  he  has  applied.  This  conclusion 
is  the  hypothesis  that  the  tutor  believes  that  the  student  is  ready  to  support. 
GUIDON'S  overlay  model  is  used  to  relate  student  behavior  to  what  MYCIN  knows:  it  is 
not  expected  that  the  rules  will  be  precise  model  of  the  student's  thinking. 
Determining  the  differences  and  the  extent  to  which  they  matter  Is  one  objective  of 
continuing  research. 

The  model  is  maintained  as  follows.  When  case  data  are  given  to  the  student, 
the  tutor  uses  the  AND/OR  tree  to  determine  the  rules  that  the  expert  has  been  able 
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to  apply,  given  this  new  information.  Thus,  domain  rules  (hereafter,  d-ru/es)  are 
applied  in  a  forward  (antecedent)  direction  to  maintain  an  exhaustive,  dynamic  record 
of  rule  applications  and  subgoal  conclusions.  Section  8.3.2.2  describes  this  in 
more  detail. 

When  the  expert  applies  a  d-rule,  special  t-rules  for  modeling  are  used  to 
decide  whether  the  student  has  also  applied  the  d-rule,  based  on  his  knowledge  of 
the  rule  and  his  ability  to  resolve  subgoals  in  the  premise.  Then,  just  as  the  tutor 
maintains  the  record  of  the  expert  program's  conclusions,  it  recursively  considers  the 
student's  use  of  this  new  knowledge  by  examining  the  d-rules  that  use  it  for  further 
modifications  to  the  knowledge  model.  Furthermore,  when  the  tutor  explicitly  tells  the 
student  about  a  rule's  application,  the  student  model  is  updated  to  reflect  the 
student's  possible  use  of  this  information.  This  model  is  called  an  "overlay"  model 
because  it  characterizes  the  student's  knowledge  solely  in  terms  of  the  expert 
knowledge  base--lt  is  an  overlay  on  the  expert  rules  [Goldstein,  1977]  [Cerr  & 
Goldstein,  1977].  (Details  and  discussion  of  limitations  appear  in  Chapter  8.) 

The  overlay  student  model  is  used  for  opportunistic  tutoring  (page  67) 
(presenting  material  to  fill  in  possible  gaps  in  the  student's  knowledge  as  opportunity 
arises  during  the  dialogue),  evaluating  a  student’s  hypothesis  (page  56)  determining 
which  d-rules  the  student  used  to  make  an  hypothesis),  and  for  guiding  the  dialogue 
(page  76)  (selecting  d-rules  and  presentation  strategies  as  a  form  of  help  for  solving 
the  problem). 

Other  components  of  the  communication  model  include  a  session  profile  (Section 
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7.1.3)  to  control  the  tutor's  remarks  (page  43),  a  focus  and  discussion  record 
(Section  7. 1.1. 2)  to  maintain  coherency,  and  a  record  of  tutorial  remarks 
(Section  7. 1.4. 3)  to  allow  the  student  to  follow  up  on  what  the  program  says 
to  him.  Finally,  GUIDON'S  t-rules  often  reference  the  lesson  plan  of  topics  that 
represent  the  program's  goals  for  the  session  (topics  and  rules  to  be  sure  to  mention). 
In  the  current  system  there  is  simply  a  single  default  strategy  that  says  a  rule  is  "on 
the  lesson  plan"  If  it  is  not  more  than  two  difficulty  levels  above  the  student’s 
sophistication  (see  Section  8.2.1). 

The  communication  model  of  GUIDON  borrows  from  theory  developed  in  natural 
language  research  (see,  for  example,  [Grosz,  1977]  [Winograd,  1977]  [Bobrow,  et 
al„  1977]),  but  makes  no  attempt  to  be  a  complete  model  of  human  interaction  in 
itself.  We  use  natural  language  research  terminology  where  appropriate  to  describe 
features  we  have  found  useful  to  incorporate  into  GUIDON. 

4.4.3  Tutorial  Rules  and  Discourse  Procedures 

Performance  knowledge  for  carrying  on  a  case  dialogue  is  represented  in 
GUIDON  as  sequences  of  rules.  Proceeding  from  an  extensive  hand  simulation, 
patterns  in  the  rule-based  tutorial  dialogues  were  formalized  into  procedures  that 
constitute  sequences  of  action  options  for  the  tutor,  and  conditions  for  controlling 
their  use.  Figure  4.9  shows  a  typical  tutorial  rule. 
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If:  1)  The  number  of  factors  appearing  in  this  rule  which  need  to  be 
asked  by  the  student  is  zero,  and 

2)  The  number  of  subgoals  remaining  to  be  determined  before  this 
rule  can  be  applied  is  equal  to  1 

Then:  Substep  i.  Say:  suggest-subgoal 

Substep  ii.  Discuss  the  goal  with  the  student  in  a 
goal-directed  mode  [ProcSOl] 

->  A  prompt  should  not  be  given  before  starting 
discussion  of  this  goal 

Substep  Iii.  Wrap  up  the  discussion  of  the  rule  being 
considered  [Proc017] 


Figure  4.9.  Tutorial  rule  for  Presenting  a  Domain  Rule 


A  range  of  tutorial  methods  have  been  formulated  in  t-rules,  consisting  of  ways 
to  discuss  a  domain  rule,  respond  when  the  student  requests  case  data,  end 
discussion  of  a  topic,  respond  to  the  student's  partial  solution  of  a  subgoal,  and  so  on. 
In  addition,  special  t-rules  are  used  to  maintain  the  overlay  model;  other  t-rules  select 
valid  question  types  when  quizzing  about  a  rule;  and  some  represent  domain- 
independent  strategies  for  diagnostic  reasoning.  These  procedures  and  tutorial  rules 
have  been  written  in  stylized  code  so  they  are  translatable  Into  an  English  form.  This 
makes  ft  easy  to  show  other  researchers  and  educators  the  specific  rhetoric 
patterns,  domain  knowledge,  and  teaching  strategies  used  by  GUIDON.  There  are 
about  45  such  procedures  containing  almost  200  t-rules  in  the  current  formalization  of 
the  case  method  dialogue.  The  formalism  and  procedures  are  described  in 
considerable  detail  in  Chapter  6,  Chapter  7,  and  Chapter  8.  The  current  set  of  t- 
rules  is  listed  in  Appendix  F. 
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To  use  GUIDON  for  teaching  students  how  to  do  diagnostic  reasoning,  we  must 
be  clear  about  what  a  student  must  learn  in  order  to  become  an  expert,  and  how  this 


relates  to  the  knowledge  that  Is  encoded  in  the  MYCIN  consultation  system.  In 
addition,  there  may  be  domain  knowledge  that  a  tutor  might  need  for  ancillary  tasks 
like  posing  valid  questions  to  the  student,  conforming  to  discourse  conventions,  and 
modeling  the  student’s  understanding.  These  considerations  should  be  of  interest  to 
other  researchers  who  wish  to  use  a  rule-based  expert  system  for  tutoring. 

This  section  considers  domain  knowledge  that  is  required  for  tutoring  that  is  not 
explicit  in  MYCIN's  rules.  The  point  of  the  exposition  Is  that  in  the  developmo'i*  -r 
MYCIN’s  knowledge  base,  using  the  simple,  uniform  rule  formalism,  certain  distinctions 
that  are  important  for  tutoring  have  not  been  made.  For  example,  it  is  possible  to 
make  distinctions  among  types  of  rule  clauses  in  order  to  account  for  why  a  rule  is 
written  the  way  it  is.  Moreover,  given  the  focus  on  "getting  the  right  answer"  during 
MYCIN's  development,  facts  and  relations  were  left  out—the  understanding  of 
processes  and  abstract  concepts  that  an  expert  knows  which  lead  him  to  believe  that 
the  rules  make  sense.  These  omissions  apparently  do  not  hinder  MYCIN's  problem¬ 
solving  competence,  but  they  severely  limit  a  tutorial  program's  ability  to  reason  about 
the  rules;  preventing  It  from  discussing  mechanistic  principles  with  the  student, 
summarizing  evidence  by  abstracting  key  points,  using  analogies  to  explain 
Inferences,  and  so  on. 

The  additional  domain  knowledge  we  have  found  useful  for  tutorials  is  of  three 
types:  (1)  SUPPORT  knowledge  (or  knowledge  that  justifies  a  rule),  (2)  STRUCTURAL 
knowledge  (patterns  in  the  rules  and  parameters  of  the  knowledge  base),  and  (3) 
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STRATEGICAL  knowledge  (meta-rules  for  controlling  lines  of  reasoning)  (see  Figure 
5.1).  The  examples  below  are  Intended  to  support  our  claim  that  transfer  of 
expertise  requires  more  than  a  regurgitation  of  problem-solving  methods,  which  is  all 
that  the  original  MYCIN  explanation  system  could  do.  The  discussion  of  strategical 
knowledge  in  Section  5.3  develops  this  point  most  thoroughly,  drawing  upon  some 
related  work  in  education  and  psychology. 


I. 

Meta-level:  STRUCTURE  4  STRATEGY 

Describes  organization  and  control  of  classes  of  rules: 

Examples:  rule  schemas  (frame) 

rule  models  (statistical  pattern) 
meta-rules  (rule) 

II. 

Object-levah  PERFORMANCE 

Used  by  MYCIN  to  solve  particular  problems : 

Rules  and  tables 

III. 

Meta-level:  SUPPORT 

Annotations  for  making  sense  of  particular  rules: 
Examples:  mechanism  descriptions 
justifications 
literature  references 

frame) 
f ixed-text) 
fixed-text) 

Figure  6.1.  Knowledge  Base  Organization  into  Three  Tiers 


Figure  6.1  places  the  meta-leveis  of  knowledge  In  relation  to  the  performance 
rule  base  that  we  described  in  Section  4.2.  We  find  it  convenient  to  group  strategical 
and  structural  knowledge  "above"  the  level  of  rules  because  this  meta-level 
knowledge  describes  the  organization  and  use  of  classes  of  rules.  In  general  we  will 
find  it  convenient  to  consider  support  knowledge  as  being  "attached"  or  specific  to 
each  rule.  However,  as  we  will  see,  mechanism  descriptions  for  the  domain  are  an 
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Important  part  of  support  knowledge  that  could  be  used  to  form  analogies  among 
groups  of  rules  as  well.  Figure  6.2  Illustrates  the  three  tiers  associated  with  a 
particular  rule.  Note  that  rule  models,  meta-rules,  and  the  author  annotation  were 
added  by  Davis  for  use  in  the  consultation  system  [Davis,  1976];  their  use  in  GUIDON 
for  tutoring  complements  Davis'  work.  The  rule  schema,  key  factor,  dual,  and 
mechanism  description  are  additions  that  we  have  made  in  the  course  of  developing 
GUIDON. 

Briefly,  the  rule  schema  and  key  factor  allow  GUIDON  to  refer  to  a  rule  without 
fully  printing  it,  as  in  summaries  of  evidence  for  an  hypothesis  (page  66).  The  dual 
rule  relationship,  which  relates  analogous  rules,  is  not  currently  used.  The  mechanism 
description  (property  of  the  key  factor)  is  used  to  produce  a  rudimentary  explanation 
of  the  rule  (Section  5.2.4).  Fixed-text  properties  are  printed  at  the  student's 
request. 

Subsequent  sections  describe  the  three  kinds  of  knowledge  that  we  have 
detected  to  be  missing  from  MYCIN's  rules  (these  follow  the  levels  structure,  strategy, 
and  support  in  Figure  6.1);  specific  examples  are  given  in  each  section.  The  section 
on  support  knowledge  goes  into  considerable  detail  to  demonstrate  the  feasibility  of 
augmenting  MYCIN's  rule  representation.  An  epilogue  considers  the  concept  of  meta¬ 
knowledge,  distinguishing  between  the  notion  of  levels  that  we  have  used  here  and 


other  uses  of  the  term. 


120 


meta-levels  of  knowledge  for  tutoring 


<  organization  and  control  ievai> 

RULE-SCHEMA:  MENINGITIS. COVERFOR. CLINICAL 
RULE-MODEL:  COVERFOR-IS-MODEL 
DUAL:  0 -RULE 57 7 


< performance  or  object  level ? 
D-RULE578 


If:  1)  The  Infection  which  requires  therapy  is  meningitis,  and 

2)  A:  A  smear  of  the  culture  was  not  examined,  or 

8:  Organisms  were  not  seen  on  the  stain  of  the  culture, 

3)  The  type  of  the  infection  is  bacterial,  and 

4)  The  patient  has  been  seriously  burned 

Then:  There  is  suggestive  evidence  (.5)  that  pseudomonas-aeruginosa  is 
one  of  the  organisms  (other  than  those  seen  on  cultures  or 
smears)  which  might  be  causing  the  Infection 

UPDATES:  COVERFOR 

USES:  (TREATINF  SPECSTAIN  EXAMSTAIN  TYPE  BURNED) 


<  support  level > 

KEY-FACTOR:  BURNED 

MECHANISM-DESCRIPTION:  (PORTAL-PATHWAY  "a  severe  burn"  "air*  "wound") 

JUSTIFICATION:  "For  a  very  brief  period  of  time  after  a  severe  burn  the 
surface  of  the  wound  is  sterile.  Shortly  thereafter,  the  area 
becomes  colonized  by  a  mixed  flora  in  which  Gram  pos  organisms 
predominate.  By  the  3rd  post  burn  day  this  bacterial  population 
becomes  dominated  by  Gram  neg  organisms.  By  the  5th  day  these 
organisms  have  Invaded  tissue  well  beneath  the  surface  of  the  burn. 
The  organisms  most  commonly  isolated  from  burn  patients  are 
Pseudomonas,  Klebslel la-Enterobacter,  and  Staph.  Infection  with 
Pseudomonas  is  frequently  fatal." 

LITERATURE:  MacMillan  BG:  Ecology  of  Bacteria  Colonizing  the  Burned  Patient 
Given  Topical  and  System  Gentamicin  Therapy:  a  five-year  study,  J 
Infect  Dis  124:278-286,  1971. 

AUTHOR:  Dr.  Victor  Yu 

LAST- CHANGE:  Sept.  8,  1976 


Figure  5.2.  Domain  Rule  578  and  Its  Associated  Documentation 

[All  information  Is  provided  by  a  domain  expert,  except  for 

the  "key-factor"  which  is  computed  by  the  tutor  from  the  rule  schema 

and  contents  of  the  particular  rule.  See  Section  5.1.4.] 
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5.1  Structural  Knowledge  —  for  organization 

GUIDON  makes  frequent  use  of  knowledge  about  the  structure  of  MYCIN'S 
knowledge.  Most  of  this  structural  knowledge  plays  no  part  in  MYCIN  consultations;  it 
has  been  added  specifically  for  Its  usefulness  in  tutoring.  This  knowledge  is 
classified  below  Into:  1)  concept  definitions,  2)  kinds  of  parameters,  3)  parameter 
clusters,  4)  kinds  of  rules,  5)  rule  clusters,  and  6)  rule  interactions.  In  some  cases, 
constructs  were  added  to  the  MYCIN  system  during  its  development  for  their 
organizational  benefit,  e.g.,  tabular  rules  (Section  6.1.3).  Other  constructs,  e.g., 
rule  schemas  (Section  5.1.4)  might  have  been  useful  during  the  knowledge 
acquisition  phase  if  they  had  been  available.  Discussion  of  the  use  of  structural 
knowledge  in  tutorials  appears  in  Section  6.2.3.I. 

5.1.1  Parameter  Separability 

Perhaps  the  first  step  In  designing  a  MYCIN-like  system  is  to  define  the 
concepts  that  will  be  named  by  the  parameters  used  in  rules.  When  parameters  are 
abstract  and  can  be  inferred  from  one  another,  it  may  be  challenging  to  keep  them 
well-defined.  Overlapping  or  partially  redundant  parameters  will  complicate  a 
student's  ability  to  make  sense  of  the  rule  set,  and  may  seriously  hinder  the  extent  to 
which  he  can  incorporate  them  in  his  own  problem-solving  approach. 

Consider  the  "Clostridia  rule"  (rule  32)  and  its  justification.  This  rule  uses 
information  about  visible  characteristics  of  an  unknown  organism  colony  growing  in  a 
culture  taken  from  the  patient  to  infer  whether  this  evidence  of  disease  is 
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"significant."  Significance  is  an  important  criterion  for  deciding  whether  to  prescribe 
therapy  to  combat  the  organism.  For  example,  if  this  is  a  naturally  occurring  organism 
that  has  contaminated  the  culture,  then  It  should  not  be  treated. 

RULE032 


If:  1)  The  site  of  the  culture  is  one  of:  those  sites  that  are  normally 
sterile,  and 

2)  The  stain  of  the  organism  is  grampos,  and 

3)  The  morphology  of  the  organism  is  rod,  and 

4)  The  aerobicity  of  the  organism  is  anaerobic 

Then:  There  is  weakly  suggestive  evidence  (.4)  that  there  is  significant 
disease  associated  with  this  occurrence  of  the  organism 


COMMENTS:  This  rule  is  directed  at  the  possibility  that  Clostridia  species 
may  be  the  pathogen.  Further  helpful  information  that  is  not  in  the 
rule  (but  should  be)  is  the  relative  size  of  the  rod  and  its 
conf iguration . 

(Clostridia  are  big  rods,  diphtheroids  are  small  and  pleomorphic.) 


Figure  6.3.  Rule  that  Exhibits  Overlapping  Use  of  Parameters 


Notice  that  this  rule  does  not  men;iou  the  organism  species  Clostridia,  yet  the 
rule  author's  comments  indicate  that  he  was  considering  the  possibility  that  a  gram¬ 
positive  anaerobic  rod  might  belong  to  the  species  Clostridia.  So  is  this  rule  about  the 
identity  of  the  organism,  Its  significance,  or  something  else? 


The  medical  Justification  for  this  rule  is  that  the  species  Clostridia  is  rarely 
found  as  a  contaminant  in  a  culture,  i.e.,  when  it  grows  in  a  culture,  it  usually  has  come 
from  the  body  site  that  was  sampled,  rather  than,  say,  introduced  to  the  sampling 
device  as  it  passed  through  the  skin  (thus  picking  up  organisms  that  are  found  on  the 
skin  surface  of  any  healthy  adult  or  child,  I.e.,  diphtheroids).  It  so  happens  that 
"contaminant"  is  another  concept  in  MYCIN.  I1"  fact,  this  rule  should  be  rewritten  to 


conclude  that  Gram-positive  rods  are  not  contaminants. 
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The  point  is  that  whether  you  are  a  student  or  an  expert  coming  to  MYCIN 
intending  to  write  new  rules,  you  will  encounter  some  difficulty  sorting  out  some  of  the 
intermediate  concepts  (see  the  subgoal  tree  structure  below). 


TREATFOR 


TREATINF  SIGNIF 

CONTAMINANT 

Figure  5.4.  Portion  of  MYCIN's  Inference  Structure 


Si^ce  there  is  a  rule  that  a  contaminant  is  not  significant,  how  do  the  two 
concepts  really  differ?  Part  of  the  problem  Is  that  prior  choices  of  concepts  and  rules 
to  include  in  the  knowledge  base  affect  later  additions.  Thus,  there  is  a  rule 

(1)  contaminant  =>  not  significant  but  not  a  rule 

(2)  not  contaminant  =>  significant. 

So  if  you  want  to  say  that  gram-positive  anaerobic  rods  require  therapy  (which  is 
what  .SIGNIFICANCE  ultimately  amounts  to),  you  can  write  a  rule  like  the  Clostridia  rule 
currently  appears  or  else  add  the  missing  rule  (2)  and  rewrite  the  Clostridia  rule  to 
conclude  "not  contaminant."  (Grampos,  anaerobic  rod  *>  not  contaminant  *> 
significant)  Indeed,  the  multiplicity  of  ways  to  get  the  same  final  result  has 
contributed  to  inconsistencies  that  one  sometimes  finds  in  the  rule  base.  Some  sort  of 
constraint  or  discipline  is  missing  that  would  make  the  definitions  of  concepts  more 
precise.  We  speculate  that  providing  explicit  Justifications  for  the  rules  according  to 
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models  like  those  described  in  Section  5.2.4  would  help  sharpen  the  rule 
authors'  use  of  concepts  in  MYCIN. 

The  sample  protocol  illustrated  how  a  student  could  be  easily  confused  about 
MYCIN's  concepts.  In  this  dialogue,  the  concept  "type  of  infection"  was  confused 
with  "name  of  infection";  the  student  told  the  experimenter  that  simply  stating  the 
possible  alternative  hypotheses  to  be  determined  would  have  helped  (i.e.,  "bacterial, 
viral,  tb,  or  fungal").1  We  have  provided  GUIDON  with  a  text  definition  file  for  many  of 
MYCIN's  parameters.  Moreover,  reprompts  used  in  a  MYCIN  consultation  to  rephrase  a 
question  for  a  user  have  been  modified  so  that  all  clarifications  are  contained  in  the 
definition  file,  allowing  this  information  to  be  used  more  readily  in  tutorials.  The 
definition  lists  examples  of  situations  when  the  concept  holds: 

**  DEFINE  TBRISK 

Definition  of:  whether  the  patient  has  a  tb  risk  factor 

The  risk  factors  for  tb  are:  a)  positive  Intermediate  strength  PPD, 
b)  history  of  close  contact  with  a  person  with  active  tb,  c)  a  household 
member  with  a  past  history  of  active  tb,  d)  apical  scarring  on  chest  xray, 
e)  history  of  a  granulomas  on  biopsy  of  liver,  lymph  node,  or  other  organs. 

Figure  6.6.  Defining  a  MYCIN  Concept  by  •  List  of  Qualifying  Criteria 

6.1.2  Parameter  Screening  Relations 

When  a  parameter  can  be  used  to  determine  the  applicability  of  one  or  more 
other  parameters  in  a  particular  case,  we  say  that  the  parameter  serves  as  a  screen 


1  GUIDON  did  this  in  Its  orientation  (page  53),  but  the  student  evidently  missed 
the  remark.  From  this  we  conclude  that  information  like  this  must  be  highlighted  and 
repeated. 
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for  the  other  parameters.  MYCIN  has  two  forms  of  screening  relationships:  those  that 
are  implicit  in  rules,  and  those  that  are  explicit  in  the  form  of  data  tables. 

Implicit  Screening  Relations 

MYCIN's  reasoning  sometimes  requires  simple  common  sense  knowledge  about 
the  world.  For  example,  one  shouldn't  ask  If  a  male  Is  "pregnant  or  breastfeeding," 
and  there  is  probably  a  (realistic)  minimum  age  for  being  an  alcoholic.  Thus,  we  say 
that  the  sex  of  the  patient  screens  "motherhood"  and  age  screens  "alcoholism." 
Unfortunately,  this  knowledge  is  not  explicit  in  the  MYCIN  system;  proper  performance 
depends  solely  on  the  ordering  of  clauses  in  rules  that  use  parameters  like 
motherhood  or  alcoholism.  The  "alcoholic  rule"  (rule535)  illustrates  this  design 
principle.  Clause  4  will  only  be  considered  if  the  patient  is  older  than  1 7. 

RULE535 

If:  1)  The  infection  which  requires  therapy  is  meningitis, 

2)  A:  A  smear  of  the  culture  was  not  examined,  or 

B:  Organisms  were  not  seen  on  the  stain  of  the  culture, 

3)  The  age  of  the  patient  is  greater  than  17  years, 

4)  The  patient  is  an  alcoholic,  and 

5)  The  type  of  the  Infection  1$  bacterial 

Then:  There  is  evidence  that  the  organisms  (other  than  those  seen  on 
cultures  or  smears)  which  might  be  causing  the  infection  is 
dlplococcus-pneumoniae  (.3)  e.coli  (.2) 

Figure  5.6.  Rule  that  Uses  Implicit  Screening 

\ 

It  is  important  for  GUIDON  to  know  about  these  relationships  so  that  it  can 
explain,  for  example,  why  MYCIN  never  considered  whether  the  patient  was  an 
alcoholic  ("because  he  ts  not  older  than  17").  To  make  these  relationships  explicit, 
the  knowledge  base  has  been  augmented  by  adding  a  relationship  "PI  screens  P2" 
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and  the  Inverse  "P 2  screened-by  PI."1  Specific  tutorial  strategies  for  using  this 
information  are  given  In  Section  6.2.2. 1. 

Explicit  Screening  Relationships!  blocks  of  data 


Blocks  of  data  are  a  second  form  of  parameter  relation.  When  MYCIN  was 
expanded  to  deal  with  meningitis  infections,  the  number  of  questions  asked  of  users 
increased  to  over  100  and  it  became  useful  to  group  questions  into  tables  for  more 
convenient  entry  of  data.  For  example,  one  table  contains  the  "findings  from  the  csf." 
Tne  property  list  representation  of  this  table  is  shown  in  Figure  6.7. 


1 

1 

(MYCIN  consultation  excerpt) 

Please  enter  CSF  findings 

in  the  following  table 

CSF 

WBC 

27)**  100 

Percent 

PMN 

99 

Proceln  CSF  Peripheral 

Glucose  Glucose 

2H  5  UNKNOWN 

CSFFINDINGS 

( part  of  MYCIN  data  structure) 

[PROP-BLOCK] 

TABPARMS:  (CSFCELLCOUNT  CSFPOLY  PROTEIN  CSFGLUC  BLOOOGLUC) 

TRANS:  (the  "results  of  tests  performed  on  the  CSF") 

SCREENED-BY: 

LP 

(added  In  GUIDON) 

Figure  5.7.  Consultation  Excerpt  end  Data  Structure  for  CSF  Findings  Block 


It  turns  out  that  there  is  a  screening  parameter  for  each  table,  e.g.,  the  csf 
findings  are  available  if  and  only  if  a  "lumbar  puncture”  has  been  performed  on  the 


1  Like  other  Information  added  to  the  system,  these  pointers  currently  reside  on 
a  special  disk  file.  We  might  modify  the  EMYCIN  representation  package  so  that  ail 
MYCIN-like  systems  routinely  Incorporate  this  Information  in  the  future. 
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patient.  Thus,  for  GUIDON'S  purposes,  we  have  added  SCREENS  and  SCREENED-BY 
relations  for  entries  in  the  table.  (E.g.,  lumbar  puncture  screens  csf  glucose,  one  of 
the  csf  findings.)  The  protocol  (page  54)  Illustrates  GUIDON'S  use  of  data  tables; 
further  discussion  of  their  tutorial  significance  appears  in  Section  7.1.1. 

5.1.3  Kinds  of  Rules 

There  are  three  primary  syntactic  kinds  of  rules;  iterative,  tabular,  and 
definitional.  These  are  domain-independent  classifications  that  GUIDON  can  discern 
by  using  templates  to  read  the  rule.  We  describe  them  here  because:  (1)  iterative 
rules  pose  special  problems  for  preparing  the  complete  AND/OR  tree,  and  require 
special  tutorial  rules  for  discussing  them;  (2)  tabular  rules  conveniently  structure  the 
rule  set,  and  it  is  easy  to  generate  Interesting  questions  from  them;  (3)  definitional 
rules  are  given  special  treatment  in  tutorials  because  they  seem  to  be  easy  to  state 
and  remember. 

Iterative  Rules 

Of  MYCIN's  450  rules,  26%  have  a  precondition  that  evaluates  a  predicate  over 

a  set  of  objects.  For  example,  "the  cutaneous  site  rule"  (rule  146)  1  shown  below, 
requires  consideration  of  all  of  the  organisms  isolated  from  positive  cultures  of  the 
patient.  The  iteration  predicate  (clause  3)  performs  a  subset  operation:  it  returns  the 
members  of  the  specified  object  set  that  satisfy  the  internal  premise  (A,  B,  and  C).2 

1  Named  with  irony,  for  the  list  of  sites  Is  not  so  described  in  the  rule  Itself. 

2  In  general,  iterative  predicates  set  a  global  variable  which  Is  passed  on  to 
other  preconditions  or  to  the  action  part  of  the  rule.  In  this  example  it  is 
COLLECTEDORGS. 
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Besides  the  subset  predicate,  there  are  two  other  kinds  of  iterative 
preconditions:  operators  to  determine  the  existence  of  a  set  and  operators  to  fund  the 
maximum  (minimum)  element  of  a  sot  (for  numeric-valued  parameters).  Appendix  A 
cicscribes  how  preconditions  of  this  type  complicate  the  process  of  configuring  a 
complete  AND/OR  tree  from  the  traces  of  a  consultation.  Section  6. 2.1.1 
d.scusses  some  of  the  problems  of  teaching  rules  of  this  type. 


RULE  146 
phrase 

If:  1)  The  infection  which  requires  therapy  is  meningitis,  and 

2)  Organisms  were  not  seen  on  the  stain  of  the  culture,  and 

3)  You  have  examined  the  organisms  isolated  from  positive  cultures 
obtained  from  the  patient,  selecting  those  for  which 

A:  The  site  of  the  culture  is  one  of:  breast-discharge  burn-site 
cutaneous-ulcer  external-ear  iv-site  rectal-fissure  surg-wouno 
trauma-wound,  and 

B:  The  identity  of  the  organism  is  known  with  certainty,  ana 
C:  There  is  significant  disease  associated  with  this  occurrence  of 
the  organism 

Then:  There  is  weakly  suggestive  evidence  (.2)  that  the  identity  of  each 
of  the  organisms  that  you  selected  is  the  organisms  (other  than 
those  seen  on  cultures  or  smears)  which  might  be  causing  me 
infection 


PREMISE:  (SAND  (SAME  CNTXT  TREAT IMF  MENINGITIS) 

( liOTSAME  CNTXT  SPECSTAIN) 

(THEREARE  (GETALL  KNOWNORG) 

(SAND  (SAME  FREEVAR  SITE 

(ONEOF  BREAST-DISCHARGE  BURN-SITE 

CUTANEOUS-ULCER  EXTERNAL-EAR 
IV-SITE  RECTAL-FISSURE 
SURG-WCUND  TRAUMA-WOUND) ) 
(DEFINITE  FREEVAR  IDENT) 

(SAME  FREEVAR  SIGNIFICANCE)) 

NIL  COLLECTEDORGS  T ) ) 

ACTION:  ( TRANSDIFPARM  COLLECTEDORGS  IDENT  CNTXT  COVERFOR  230) 

COMMENTS:  This  rule  uses  information  about  nonsterile  cutaneous  sites  to 
determine  what  might  be  causing  the  meningitis.  (See  also  Rule  153.) 
Created:  l-JUL-77  12:29 
Last  edited:  23-SEP-77  14:54 


Figure  3.8.  An  Interative  Rule:  The  Cutaneous  Site  Rule 


Structural  Knowledge  —  for  organization 


129 


Tabular  Rules 

About  10  percent  of  MYCIN's  meningitis  rules  have  a  table  for  computing  the 
conclusion  of  the  action  part.  "The  CSF  Glucose  Rule"  (Rule  602)  Is  typical  (Figure 
5.9).  Notice  that  the  table  indexes  the  certainty  factors  for  a  list  of  conclusions, 
based  on  the  range  of  the  csf  glucose  value.  Rules  of  this  kind  were  placed  in  the 
system  In  order  to  collapse  the  rule  set;  the  rule  authors  and  system  maintainors 
found  this  representation  to  be  more  convenient  than  using  a  separate  rule  for  each 

range  of  the  csf  glucose  value.1  The  use  of  tabular  rules  is  an  important  means  of 
structuring  the  knowledge  base,  and  the  organizational  benefit  is  at  least  as  important 
for  tutoring  the  rules,  as  for  the  original  acquisition  process. 

It  is  fairly  easy  to  generate  multiple  choice  and  fill-in  questions  from  tabular 
rules.  Three  examples  are  shown  In  Figure  6.10  .  Preceding  each  example  is  the 
specification  given  to  the  question-generating  routine.  Appendix  B  describes  the 
process  of  generating  questions  from  tabular  rules.  One  of  the  complications  is  that 
"incorrect  conclusions”  cannot  simply  be  chosen  randomly  from  the  conclusion  table— 
the  program  must  be  sensitive  to  what  is  a  meaningful  change  in  the  correct  certainty 
factor. 


1  Tabular  rules  were  developed  by  Lawrence  Fagan  for  the  PUFF  system.  In 
PUFF,  1 6  percent  of  the  rules  are  tabular. 
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RULE502 


If:  1)  The  infection  which  requires  therapy  is  meningitis, 

2)  A  lumbar  puncture  has  been  performed  on  the  patient, 

3)  The  csf  glucose  value  is  less  than  or  equal  to  40,  and 

4)  A:  The  simultaneous  blood  glucose  value  is  not  known,  or 

6:  The  csf  glucose  value  divided  by  the  simultaneous  blood  glucose 
value  is  less  than  or  equal  to  .4 
Then:  The  type  of  the  infection  is  as  follows: 

If  the  CSF  glucose  value  is: 

a)  less  than  5  then:  bacterial  (.4),  not  viral  (.4),  not  fungal 

(•?), 

b)  between  5  and  10  then:  bacterial  (.35),  not  viral  (.35), 

c)  between  10  and  15  then:  bacterial  (.3),  not  viral  (.3),  fungal 
(.3),  tb  (.3); 

d)  between  15  and  20  then:  bacterial  (.25),  fungal  (.25),  tb  (.25); 

e)  between  20  and  40  then:  bacterial  (.2),  fungal  (.2),  tb  (.2); 


PREMISE :  (SAND  (SAME  CNTXT  TREATINF  MENINGITIS) 

(SAME  CNTXT  LP) 

(LESSEQ*  ( VAL1  CNTXT  CSFGLUC) 

40) 

(SOR  (NOTKNOWN  CN1X1  BLOODGLUC) 

( LESSEQ*  (FQUOTIENT  (VAL1  CNTXT  CSFGLUC) 

( VAL 1  CNTXT  BLOODGLUC)) 

•  4)5) 

ACTION:  (CONCLUDET  CNTXT  (VAL1  CNTXT  CSFGLUC) 

’ ( (LT  5  400  -400  -200  0) 

(BT  5  10  350  -350  0  0) 

(BT  10  15  300  -300  300  300) 

(BT  15  20  250  0  250  250) 

(BT  20  40  200  0  200  200)) 

TALLY  TYPE  ’(BACTERIAL  VIRAL  FUNGAL  TB)) 


Figure  6.9.  A  Tabular  Rulei  Tha  CSF  Glucosa  Rule 
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PREMISE:  multiple  choice 
ACTION:  actual  conclusion 

Given  the  fact  that  the  simultaneous  blood  glucose  value  is  not  known,  what 
value  for  the  CSF  glucose  value  would  tell  you  that  the  type  of  the 
infection  is  bacterial  (.35)  viral  (-.35)? 

Please  choose  one  of  the  following  (1  to  5): 

1)  less  than  5 

2)  between  5  and  10 

3)  between  10  and  15 

4)  between  15  and  20 

5)  between  20  and  40 

**  4 

No,  because  the  CSF  glucose  value  Is  between  5  and  10  we  can  conclude  the 
type  of  the  Infection  is  bacterial  (.35)  viral  (-.35)  [RULE502]. 

If  It  were  true  that  the  CSF  glucose  value  were  between  15  and  20  then  one 
could  conclude  that  the  type  of  the  infection  is  bacterial  (.25)  tungal 
( .25)  tb  ( .25). 


PREMISE:  case  facts 
ACTION:  multiple  choice 

From  the  CSF  glucose  value  and  the  fact  that  the  simultaneous  blood  glucose 
value  Is  not  known  one  can  conclude  that  the  type  of  the  Infection  is 
(select  from  the  following): 

1)  fungal  --  strongly  suggestive  evidence  (.92) 

2)  bacterial  --  weakly  suggestive  evidence  (.35) 

3)  not  viral  --  weakly  suggestive  evidence  (-.35) 

4)  tb  --  weakly  suggestive  evidence  (.08) 

**  2  4 

The  following  value  is  correct:  bacterial  (.35). 

The  following  value  is  not  correct:  tb  (.08). 

The  following  value  Is  missing  from  your  answer:  viral  (-.35). 

(Refer  to  RULE 502. ) 


PREMISE:  case  facts 
ACTION:  incorrect  conclusion 

Given  the  CSF  glucose  value  and  the  fact  that  the  simultaneous  blood  glucose 
value  Is  not  known,  does  this  tell  you  that  the  type  of  the  Infection  is 
bacterial  (.4)  viral  (-.4)  fungal  (-.2)? 

**  YES 

No,  given  these  facts,  we  can  conclude  that  the  type  of  the  infection  is 
bacterial  (.35)  viral  (-.35)  [RULE502]. 

The  conclusions  in  the  question  hold  when  the  CSF  glucose  value  Is  less  than 
5. 

Figure  6.10.  Three  Semple  Questions  Generated  from  e  Tabular  Rule 

(shown  in  Figure  6.0) 
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Definitional  Rules 


Some  of  MYCIN's  rules  state  common  sense  relations  that  shouldn't  need  to  be 
explained  to  a  student.  An  example  is  the  rule  (paraphrased)  that  "the  duration  that 
a  drug  was  prescribed  is  the  time  elapsed  from  when  it  was  started  to  when  it  was 
discontinued."  At  first,  it  appeared  that  GUIDON  could  recognize  all  such  rules  by 
presence  of  a  definite  CF  (1.0)  in  a  rule's  action.  That  is,  these  rules  are  definitional . 
However,  it  soon  became  clear  that  a  rule  is  definitional  to  a  student  only  if  he  has 
the  necessary  background  knowledge  to  support  it  "with  certainty."  Moreover,  there  is 
such  a  thing  as  "definitional  intent”  that  has  little  to  do  with  how  certain  the  rule 
author  was  about  the  strength  of  evidence--a  somewhat  unexpected  distinction. 


We  concluded  that  there  are  three  kinds  of  rules  that  determine  values  with 
certainty: 


1)  Common  sons*  rules  that  the  man-on-the-street  should 
know,  e.g.,  "if  you  didn't  find  organisms  on  the  culture  when  you 
looked  under  the  microscope,  then  re  are  no  organisms  to 
identify." 


2)  Basic  medicine  rules  that  we  can  expect  a  second  year 
medical  student  to  know  (though  we  may  find  It  useful  to  make  finer 
,  distinctions)  e.g.,  “if  the  route  of  administration  for  the  drug  was 
«  oral,  and  the  drug  Is  one  that  is  poorly  absorbed  in  the  Intestinal 
tract,  then  the  dose  of  the  drug  received  by  the  patient  is 
inadequate."  Mo*t  definitional  rutea  are  of  this  type. 


3)  Intent  rules  that  advocate  a  definite  therapeutic  choice  of 
action,  e.g.,  "if  the  patient  is  less  than  8  years  old  then  don't 
prescribe  tetracycline."  (Rules  like  this  frequently  appear  to  make  a 
great  leap  from  premise  to  conclusion.  The  problem  of  explaining 
them  to  a  student  is  considered  in  Section  6.2.) 
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Originally,  we  had  hoped  that  the  student  model  could  be  Initialized  by  assuming 
that  every  student  would  know  every  definitional  rule.  Given  the  findings  outlined 
above,  it  became  necessary  to  examine  each  rule  individually.  (The  process  of 
setting  up  the  a  priori  component  of  the  student  model  is  discussed  further  in  Section 
8.2.1.)  Nevertheless,  we  found  It  useful  to  write  special  discourse  procedures 
for  dealing  with  definitional  rules  because  there  are  so  many  of  thom  «i  they  seem 
to  be  easily  remembered  by  new  students.  Section  6.2.1. 2  describes  these 
discourse  procedures. 

5.1.4  Rule  Clusters 

Rule  Models 

Rule  models  [Davis,  1976]  are  program-generated  patterns  that  represent 
typical  clusters  of  factors  in  rule  preconditions  for  determining  a  given  parameter. 
Rule  models  do  not  necessarily  correspond  to  domain  concepts,  though  in  so  far  as 
correlation  of  factors  is  an  indication  of  Importance,  they  usually  represent  general 
principles  that  are  part  of  the  human  expert’s  organization  of  his  knowledge.  An 
example  is  shown  below;  It  indicates  that  the  name  of  the  infection  (TREATING)  and 
the  subclassification  of  the  infection  (TYPE)  tend  to  appear  together  in  rules  for 
determining  the  organisms  that  should  be  treated  for,  based  on  clinical  evidence 
alone.1 


1  These  patterns  are  simply  pairwise  correlations  between  all  parameters  that 
appear  In  a  rule  group.  The  numbers  are  computed  by  summing  the  certainty  factors 
that  appear  In  rule  conclusions,  and  thus  reflect  a  measure  of  frequency  of 
appearance  and  strength  of  evidence  [Oavis,  1976].  The  entry  designated  by  an 
arrow  (— »  was  used  by  GUIDON  in  the  sample  Interaction  when  presenting  orientation 
for  pursuing  this  goal  (page  47). 
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COVERFOR-MOD 
P-AOVICE:  (j 

( 

— >  ( 

( 


TREATINF  SAME  40068) 

TYPE  SAME  22500 ) 

SPECSTAIN  NOTSAME  13408) 

((TREATINF  SAME)  (TYPE  SAME)  40008) 
(TYPE  SAME)  (TREATINF  SAME)  22500) 
(SPECSTAIN  NOTSAME)  (TREATINF  SAME) 
(TYPE  SAME)  (EXAMSTAIN  NOTSAME)  13400) 
(PENDCUL  SUBJECT)  (TREATINF  SAME) 

TYPE  SAME)  (SPECSTAIN  NOTSAME) 
EXAMSTAIN  NOTSAME)  10900) 

NONCUL  SUBJECT)  (TREATINF  SAME) 

(TYPE  SAME)  (SPECSTAIN  NOTSAME) 

I  EXAMSTAIN  NOTSAME)  9700) 


A-ADVICE-.  ((COVERFOR  CONCLUDE  CONCLUDE*  42800) 
((COVERFOR)  (PENDCUL)  (NONCUL) 
(SMEARORG)  (KNOWNORG)  42008) 
(PENDCUL  SUBJECT) 

(COVERFOR  CONCLUDE  CONCLUDE*)  10900) 


Figure  6.1 1.  Rule  Model  for  the  Parameter  COVERFOR 


Because  rule  models  make  explicit  the  factors  that  most  commonly  appear  in 
rules  for  pursuing  a  goal,  we  are  using  them  as  a  form  of  orientation  for  beginning 
students  (Section  6.2.3.1).  For  GUIDON'S  purposes,  rule  models  have  been 
altered  to  include  the  context  of  application  of  the  rule,  namely  the  kind  of  object  in 

the  context  tree  to  which  the  rule  applies,  e.g.,  "pending  cultures."1  For  reasons  that 
will  become  clear  below,  this  provides  a  much  more  useful  analysis  of  patterns  in  the 
rules 


1  This  Is  done  by  simply  Including  a  rule's  subject  of  application  as  a  precondition 
(which  it  is  in  practice),  with  the  "predicate"  SUBJECT.  This  extension  to  Randy 
Davis'  rule  model  creation  code  was  made  in  collaboration  with  Jim  Bennett. 


1 


Structural  Knowledge  —  for  organization 


136 


Rule  Schemas 


A  rule  schema  is  a  description  of  a  kind  of  rule:  a  pattern  of  preconditions  that 
appears  in  the  premise,  the  goal  concluded  about,  and  the  context  of  its  application. 
Unlike  rule  models,  these  schemas  are  given  to  the  program  maintalners  by  a  physician 
expert.  An  example  is  shown  in  Figure  5.12;  notice  that  provision  is  made  for  relating 
analogous  schemas. 


COVERFOR .CLINICAL  [ PROP-RULE . SCHEMA ] 


ANALOGOUS-SCHEMA :  ( COVERFOR . LABDATA ) 

CF-COMPARISON :  GREATER 

IMPORT:  ("When  trying  to  find  out  the  organisms  that 
require  therapy,  give  substantial  weight  to 
clinical  evidence  when  there  Is  no  (hard)  lab 
data  available") 

PARMS-SCHEMA:  (TREATINF  EXAMSTAIN  SPECSTAIN  TYPE) 
PREM-SCHEMA:  (SAND  (SAME  CNTXT  TREATINF  MENINGITIS) 

(JOR  (NOTSAME  CNTXT  EXAMSTAIN) 
(NOTSAME  CNTXT  SPECSTAIN)) 

(SAME  CNTXT  TYPE  BACTERIAL)) 
SUBJ-SCHEMA:  NONPOSRULES 
UPDATE-PARM:  COVERFOR 


Figure  5.12.  Rule  Schema  for  the  Parameter  COVERFOR 


Schemas  were  introduced  to  the  system  when  it  was  observed  that  the  expert 
had  written  large  sets  of  rules  by  working  from  a  template  or  pattern.  Each  rule  in  the 
set  had  a  fixed  set  of  introductory  clauses  and  one  or  more  factors  usually  specific  to 
that  rule.  Using  the  template  (rule  schema),  GUIDON  can  "subtract  off"  the  rule 
preconditions  common  to  all  rules  of  the  type,  leaving  behind  the  factors  that  are 
specific  to  the  particular  rule  (this  is  done  dynamically,  during  the  tutorial).  These  are 
called  the  key  factors  of  the  rule.  Thus,  the  key  factor  of  d-rule  578  (see  Figure 
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6  2),  "the  patient  was  seriously  burned,"  was  determined  by  removing  the  contextual 
information  of  the  name  of  the  infection,  whether  organisms  were  seen,  and  the  type 
of  the  infection. 


When  computing  the  key  factors  of  a  rule,  GUIDON  also  removes  restrictions 
specified  by  screening  relations  (Section  6.1.2)  and  exception  clauses  which  are 
generally  present  to  screen  for  a  condition  that  is  dealt  with  in  a  special  rule  (e  g,  if 
the  patient  has  a  head  injury,  default  information  based  on  the  age  of  the  patient 
alone  is  not  useful  (Figure  6.13)). 

RULES07 


If:  1)  The  Infection  which  requires  therapy  Is  meningitis, 

2)  A:  A  smear  of  the  culture  was  not  examined,  or 

8:  Organisms  were  not  seen  on  the  stain  of  the  culture, 

3)  The  type  of  the  infection  Is  bacterial, 

4)  The  patient  does  not  have  a  head  injury  defect,  and 

5)  The  age  of  the  patient  is  known 

Then:  The  organisms  (other  than  those  seen  on  cultures  or  smears)  which 
might  be  causing  the  infection  Is  as  follows: 

If  the  age  of  the  patient  is: 

a)  less  than  10  days  then:  e.coll  (.75),  klebsiel la-pneumoniae 
(.5),  streptococcus-group-b  (.75), 

b)  between  10  days  and  1  month  then:  e.coli  (.75),  klebsiel la- 
pneumoniae  (.5),  streptococcus-group-b  (.75), 

c:)  between  l  month  and  2  months  then:  e.coli  (.75),  streptococcus- 
group-b  (.75),  listeria  (.4); 

d)  between  2  months  and  1  year  then:  diplococcus-pneumoniae  (.41), 
streptococcus-group-b  (.4),  hemophilus-inf luenzae  (.75), 
neisser la-menlngitidis  (.4), 

e)  between  1  year  and  5  years  then:  diplococcus-pneumoniae  (.41), 
hemophl lus-lnf luenzae  (.75),  neisser la-menlngltidls  (.4), 

f)  between  5  years  and  15  years  then:  diplococcus-pneumoniae  (.75), 
hemophllus-lnfluenzae  (.73),  neisser Ia-mening f'.idis  (.74), 

g)  between  15  years  and  55  years  then:  diplococcus-pneumoniae 
(.75),  neisser ia-meningitldls  (.74), 

h)  greater  or  equal  to  55  years  then:  staphylococcus-coag-pos  (.4), 
streptococcus-species  (.4),  diplococcus-pneumoniae  (.75), 


SCHEMA:  COVERFOP  CLINICAL 
EXCEPTIONS:  HEAOiNJOEF 
DUAL:  RULE554 


Figure  6.13.  The  Age  Rule  and  Its  Scheme-related  Properties 
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Examples  of  the  use  of  key  factors  occur  throughout  the  sample  protocol,  e.g.,  on 
page  76,  when  GUIDON  says,  "...considering  whether  J.Smlth  has  a  head  injury  defect 
[RULE607]." 

As  shown  in  Figure  6.12,  there  are  two  COVERFOR  rule  schemas  that  are  duals. 
Dual  schemas  der-irtbe  two  classes  of  rules  which  have  the  property  that  each  rule  in 
one  class  has  an  analogue  (dual)  In  the  other.  Making  the  relationship  between  the 
classes  in  rules  explicit  (in  the  schema  descriptions),  would  make  it  possible  for  the 
tutorial  program  to  form  analogies  between  specific  rules.  For  example,  after  the 
tutor  and  student  have  discussed  rule  607,  the  tutor  could  ask  the  student  what 
conclusions  he  would  make  If  he  had  laboratory  data,  as  well  as  clinical  information  for 
making  a  diagnosis,  thus  asking  for  the  student  to  state  the  specific  rule  for  an 
analogous  situation.  Under  this  notion  of  analogy,  two  specific  situations  are 
analogous  if  the  classes  to  which  they  belong  (that  is,  their  templates)  are 
analogous.1  Observe  that  this  kind  of  knowledge,  meta-knowledge  that  makes  rule 
relationships  explicit,  enables  a  tutor  to  teach  general  concepts  as  well  as  specific 
inferences.  We  intend  to  implement  strategies  for  using  dual  rule  schemas  in  later 
versions  of  the  system. 

The  duals  discussed  here  are  contextual;  other  duals,  much  more  difficult  to 
detect  by  inspection,  involve  exception  clauses.  For  example,  there  is  a  rule  that 
mentions  cancer  cases  (i.e.,  Lymphoma  and  Leukemia)  that  is  a  dual  to  a  rule  that 
restricts  itself  to  non-cancer  cases.  The  second  rule  mentions  steroids,  but  the  first 


1  Using  the  same  notion  of  analogy,  Goldstein  has  recently  added  links  between 
analogous  rules  In  the  skill  network  for  the  WUMPUS  game  [Goldstein,  1978]. 
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does  not  because  the  rule  author  assumes  that  a  cancer  patient  will  be  receiving 
steroids.  Thus,  this  pair  of  rules  implicitly  considers  the  effect  of  steroids  with  and 
without  cancer.  (See  "programming  with  rules"  in  Section  6.1.6.) 


Subgoal  Structure 


Another  way  to  organize  the  rules  is  to  make  explicit  the  structure  of  subgoals. 
This  construct  may  provide  a  valuable  foundation  for  conveying  the  rule  base  to 
students,  for  the  general  outline  of  MYCIN's  AND/OR  tree  corresponds  quite  well  to 
the  approach  that  is  often  recommended  to  medical  students  (for  example,  see 
[Harrison’s,  1974]  [Yu,  et  al.,  1978]  ).  Figure  6.14  illustrates  the  subgoal  structure 
of  f.iYCIN's  rules.1 


REGION 


TREATFOR 


TREATINF  SIGNIF 


COVERFOR 


CONTAMINANT 


Figure  6.14.  Subgoal  Structure  of  MYCIN's  Rule  Set 


GUIDON  conveys  this  structure  to  a  student  in  three  ways  during  a  tutorial: 


1 )  A  sketch  of  the  subgoal  tree  (generated  by  a  procedure  from 
the  AND/OR  tree)  lays  out  the  "most  Important"  steps  for  reaching  a 


1  Bennett  has  called  such  a  construct  an  inference  structure  and  has  described  its 
importance  in  knowledge  acquisition  [Bennett  &  Engelmore,  1 979]. 
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diagnosis  in  a  particular  case  (see  Appendix  D  and  Section 
6.2.3. 1). 


2)  Rule  models  reveal  one  level  of  the  tree;  they  are 
presented  when  discussion  of  a  topic  begins  and  are  independent 
of  any  particular  case  (see  orientation  strategies,  Section 
6.2.3. 1 ). 


3)  Strategies  for  assisting  the  student  single  out  subgoals 
that  haven't  been  completed  and  suggest  them  to  the  student.  (See 
page  76  and  Section  6.2. 3.2  for  discussion.) 


5.1.6  Rule  Interactions 

In  this  section  we  will  consider  the  effects  that  rules  have  on  one  another.  The 
main  thrust  will  be  to  describe  how  the  rule  formalism  has  been  used  to  encode 
knowledge  to  get  a  "performance  effect,"  yet  no  meta-knowledge  annotates  these 
decisions.  Consequently,  the  tutor  is  unaware  of  many  subtle  nuances  that  lie  behind 
choices  like  the  certainty  factor  of  a  rule's  conclusion,  whether  or  not  a  rule  is  "self- 
referencing"  (concludes  about  a  parameter  mentioned  In  the  premise),  and  how  a 
rule's  dual  Is  encoded.  We  have  dealt  with  some  of  these  problems  by  adding  meta¬ 
knowledge  to  the  system,  in  other  cases  only  a  fixed-text  justification  annotates  the 


rule. 
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The  Cascading  Certainty  Factor  Problem 

To  compute  the  certainty  of  the  conclusions  in  a  rule's  action,  MYCIN  multiplies 
the  minimum  certainty  of  the  set  of  preconditions  (passed  as  the  variable  TALLY)  by 
the  certainty  factor  (CF)  that  appears  in  the  rule's  action.  This  CF  is  then  combined 

with  those  derived  from  other  rules  that  successfully  update  the  parameter;  see 
Figure  6.15. 


Rule  R1  =>  Para  PI 

PI  used  in  Rule  R 2  *>  Pare  92 

Parameter  CFs  are 

92  used  in  . 

modified  by  each 

• 

rule's  CF. 

* 

Ask=>CF  [Rule  Rx  +  Rule  Rn  +  ...  +  Rule  Ry]=>CF 

l  a 

1 

|  Final  CF  combines 

Final  CF  for  Para 

>  user's  certainty 

}  with  deduced  CF 

Figure  6.16.  CF  Combinations  in  the  AND/OR  Tree 
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For  the  resultant  conclusion  (the  combination  of  all  CFs  in  rules  that  succeed)  to 
be  considered  as  evidence,  it  must  exceed  .2.  For  example,  if  a  rule  precondition 
states  "the  morphology  of  the  organism  is  rods,"  the  CF  associated  with  rods  must 
exceed  .2  for  this  precondition  to  succeed.  The  cascading  problem  arises  when  belief 
is  handed  down  in  a  hierarchy  of  rules:  As  CFs  are  computed  at  each  level,  the  CF 
being  pushed  to  the  top  of  the  tree  becomes  smaller  and  smaller  because  each  level 
multiplies  the  result  by  a  number  less  than  1.  See  Figure  5.16. 


level 

Parameter 

Minimum  Rule  CF 

Minimum  Conclusion  CF 

CF  CUTOFF 

1 

TREATID 

.4 

.5  x  .4  *  .2 

.2 

2 

COVERFOR 

.7 

.7  x  .7  e  .49 

.5 

3 

TYPE 

.7 

.7 

.7 

Figure  6.16.  Table  of  CF  Thresholds  In  the  AND/OR  Tree 
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This  table  illustrates  that,  as  we  descend  into  the  AND/OR  tree,  rules  must  use 
higher  certainty  factors  in  order  for  the  result  at  the  top  of  the  tree  to  exceed  the  .2 
cutoff.  Specifically,  a  rule  that  concludes  COVERFOR  must  use  CFs  greater  than  .7 
("minimum  rule  cf")  so  that  when  TALLY  reflects  the  cumulative  CF  for  TYPE  (the 
"minimum  conclusion  cf,"  also  .7),  the  result  will  be  greater  than  .49.  Then,  when  the 
cumulative  CF  for  COVERFOR  is  at  least  .5,  and  it  is  used  in  TREATID  rules  having  a 
rule  CF  greater  than  .4,  the  final  outcome  will  be  greater  than  .2,  the  cutoff  for 
"significant  evidence."  Put  another  way,  if  you're  writing  a  rule  that  concludes  that 
Pseudomonas  should  be  treated  (COVERFOR  pseudomonas),  then  you'll  need  to  use  a 
CF  greater  than  .7  in  the  rule,  to  allow  for  the  reduction  by  TALLY  coming  from  TYPE 
and  tne  expected  change  higher  in  the  tree  when  COVERFOR  Is  reduced  by  the 
TREATID  rules. 

The  column  on  the  far  right,  "CF  CUTOFF,"  is  the  CF  threshold  for  cumulative 
evidence  on  each  level.  It  is  used  by  GUIDON  to  provide  commentary  to  the  student 
to  inform  him  about  the  changing  Interpretation  of  CFs  throughout  the  tree.  For 
example,  if  the  tutor  and  student  have  just  gathered  evidence  to  support  the 
hypothesis  that  Pseudomonas  should  be  covered  for,  and  the  cumulative  CF  is  below 
.6,  GUIDON  will  tell  the  student  that  the  cumulative  evidence  is  not  yet  significant 
(analogous  to  what  occurs  on  page  76). 

Programming  With  Rules 

Sometimes  single  rules  in  isolation  may  appear  nonsensical  even  to  another 
expert,  but  when  their  effect  is  combined  with  other  rules,  the  consultation  system 
performs  correctly.  Some  examples  are  given  here. 
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1 )  The  fungal  rules.  There  are  basically  two  fungi  that  cause 
fungal  meningitis;  the  rule  set  Is  designed  so  that  a  single  rule 
always  posits  that  both  fungi  are  present  and  other  rules 
"discredit"  one  fungus  when  there  is  evidence  for  the  other.  For 
example,  if  the  patient  was  in  an  area  that  is  endemic  to 
coccidiomycoses,  then  cryptococcus  (the  other  fungi)  is  concluded 
with  a  negative  C F. 


2)  The  partial  treatment  rule.  When  a  patient  with  bacterial 
meningitis  has  been  receiving  antibiotics  in  the  period  before  the 
consultant  was  called  in,  the  CSF  findings  are  shifted  (e.g.,  the  CSF 
glucose  value  increases),  so  that  the  evidence  is  not  as  strong  that 
the  infection  is  bacterial.  Consequently,  rules  that  use  the  CSF 
findings  will  be  fooled  Into  concluding  "not  bacterial,"  while  in  fact 
the  patient  is  simply  recovering  from  his  bacterial  infection. 


Rather  than  using  the  fact  that  the  meningitis  is  partially 
treated  as  contextual  information  to  modify  each  rule  that 
interprets  CSF  findings,  the  rule  authors  chose  to  write  a  single  rule 
that  says,  "if  the  meningitis  is  partlally  treated,  then  increase  your 
belief  that  the  infection  Is  bacterial.”  Thus,  this  rule  adjusts  the 
certainty  of  a  conclusion  to  get  the  right  effect.  However,  in 
isolation  the  rule  makes  no  sense:  A  naive  student  might  conclude 
that  antibiotics  somehow  cause  bacteria)  meningitis  (!). 


3)  The  x-Ray  Rules.  Rules  can  be  Implicit  duals  of  one 
another,  without  any  indication  in  the  knowledge  base  that 
complimentary  situations  are  being  considered.  For  example,  there 
Is  a  rule  that  says  that  when  the  chest  x-ray  is  normal  and  the 
patient  does  not  have  a  tb  risk  factor,  then  don't  worry  about  a  TB 
infection.  Another  rule  says  that  an  abnormal  chest  x-ray  that 
suggests  Active-tb  is  reason  for  considering  TB. 


These  two  rules  are  interesting  because  one  is  (almost)  the 
inverse  of  the  other.  Here  a  simple  Idea  has  been  split  into  two 
parts,  with  no  trace  of  the  connection  to  be  found.  When  a  tutorial 
program  looks  around  for  interesting  questions  to  pose  to  a  student, 
it  needs  special  heuristics  to  find  pairs  of  rules  like  this.  (Suppose 
the  x-ray  is  normal  and  there  is  no  tb-risk,  the  tutor  might  want  to 
ask,  "If  the  x-ray  were  abnormal  and  Indicated  active  tb,  what 
organism  might  be  causing  the  Infection?"  )  A  similar  relationship 
exists  among  the  rules  for  concluding  whether  a  culture  was  taken 
from  a  sterile  source  in  the  body.  The  rule  authors  designed  the  set 
of  eight  rules  so  that  they  are  disjoint:  exactly  one  rule  succeeds 
in  every  case. 
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There  appears  to  be  a  choice  here:  write  heuristics  for 
"discovering"  these  patterns  by  analyzing  the  rule  base  (preferably 
one  set  of  modular  functions  or  rules  for  use  by  the  knowledge 
acquisition,  explanation,  and  tutorial  programs),  or  make  the 
relationships  explicit  in  the  representation. 


4)  Self-referencing  Rules.  The  only  way  to  control  the  order 
that  rules  are  applied  in  MYCIN  is  to  make  some  rules  "self* 
referencing."  These  are  rules  for  determining  a  parameter  that  refer 
to  the  parameter  being  concluded  about.  For  example,  the  rules  for 
concluding  the  name  of  the  infection  (meningitis,  bacteremia,  etc.) 
are  broken  Into  two  groups:  (1)  normal  rules  that  seek  to  discredit 
the  default  conclusion,  and  (2)  "self-referencing  rules"  that  make  a 
default  conclusion  in  the  absence  of  negative  evidence  from  the 
first  group  (for  example,  one  self-referencing  rule  states  that  "if 
there  is  no  evidence  that  meningitis  is  not  the  infection  and  the  site 
cultured  is  CSF,  then  assume  that  the  infection  is  meningitis"). 


Interactions  like  this  require  the  tutorial  program  to  be 
sensitive  to  whether  or  not  a  rule  is  self-referencing.  In  particular, 
no  self-referencing  rule  for  a  parameter  can  properly  be  considered 
for  discussion  until  all  of  the  normal  rules  for  this  parameter  have 
been  evaluated.  The  overlay  model  (Section  8.3.2)  enforces 
this  distinction,  however  the  tutor  sometimes  finds  it  convenient  to 
ignore  normal  rules  that  fail  to  apply  to  the  particular  case,  and  this 
generally  causes  no  difficulties. 


In  summary,  rules  do  not  exist  In  isolation  to  one  another;  rule  authors  were 
often  quite  aware  of  interactions  that  ensured  correct  results.  None  of  these 
interactions  are  explicit  in  the  MYCIN  knowledge  base.  We  have  given  GUIDON  some 
ad  hoc  meta-knowledge  about  this  structure,  but  most  of  it  can  only  be  discovered 
and  understood  by  reading  the  fixed-text  comments  and  justifications  written  by  rule 
authors.  It  is  perhaps  inevitable  that  every  knowledge  representation  we  use  will 
have  an  accompanying  folklore  for  its  use.  The  challenge  Is  to  make  this  Information 
explicit,  and  to  keep  its  need  to  a  minimum. 
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5.2  Support  Knowledge  —  for  explelning  rules 

One  MYCIN  rule  says,  "If  the  patient  is  less  than  8  years  old,  don't  prescribe 
tetracycline."  As  a  dictum,  this  is  clear  enough,  and  it  indeed  makes  MYCIN  perforin 
properly  when  prescribing  drugs  for  youngsters.  However,  unless  you  know  about 
chelation,  the  process  of  drug  deposition  in  developing  bones,  and  the  blackened 
permanent  teeth  that  result,  you  won't  be  able  to  understand  why  this  rule  is 
medically  (and  socially)  correct.  One  of  the  points  we  will  develop  here  is  that  after 
you  have  understood  a  number  of  rules  like  this,  you  begin  to  form  an  intuition,  or 
model  for  the  kinds  of  arguments  that  lie  behind  rules  of  this  type.  So  after  learning  a 
few  of  of  MYCIN's  drug  contraindication  rules,  like  the  dictum  about  Tetracycline,  you 
will  find  it  natural  to  group  them  under  some  rubric  like  "undesirable  body  changes  of 
antibiotics  that  depend  on  patient-specific  factors."  Accompanying  this  concept  will 
be  the  notion  of  a  process,  In  this  case  telling  the  story  of  drug  administration  by  some 
method,  migration  of  the  drug  In  the  body,  chemical  effects  along  the  way,  deposition, 
undesirable  body  changes,  and  so  on.  We  claim  that  mechanistic  stories  like  this  form 
the  foundation  for  satisfying  explanations  of  MYCIN's  rules:  they  support  the  leap  from 
premise  to  conclusion. 

The  subsections  below  characterize  MYCIN's  rules  as  optimized  knowledge 
(Section  6.2.1)  and  then  categorize  the  forms  of  knowledge  that  are  not 
expressed  in  the  rules,  namely  principles  of  abstraction,  inference,  and  mechanism 
(Section  6.2.2).  Implications  for  tutoring  are  considered  (Section  5.2.3). 
Mechanism  descriptions  are  introduced  as  a  form  of  meta-knowledge  that  GUIDON 
uses  to  explain  rules  (Section  6.2.4).  It  is  argued  that  underlying  process 
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models  appear  to  be  a  form  of  expert  knowledge  that  is  orthogonal  to  the  set  o( 
performance  rules,  posing  some  strict  limitations  on  GUIDON'S  ability  to  discuss  and 
relate  individual  rules  when  these  models  are  not  available. 

6.2.1  What's  in  a  Rule? 

The  most  serious  hindrance  to  using  MYCIN's  rules  for  teaching  is  that  significant 
reasoning  steps  are  ''missing'1  from  nearly  every  rule.  Put  simply,  MYCIN's  rules 
represent  optimned  knowledge.  Domain  relations  that  clinch  the  inference  argument  of 
the  rule  have  been  "compiled  out"-,  they  are  not  explicit  anywhere  in  the  knowledge 
base.  If  a  student  does  not  know  these  relationships  or  has  forgotten  them,  then  he 
will  not  be  able  to  make  sense  of  the  rule.  However,  the  expert  who  wrote  the  rule 
did  not  find  it  necessary  to  include  these  steps  because  evidently  their  applicability 
does  not  vary  from  case  to  case.  That  is,  there  is  no  need  to  parametrize  this 
knowledge--to  make  it  dependent  on  other  rules  or  to  ask  the  program's  user  to  verify 
that  they  hold. 

For  example,  observe  that  chelation  of  drugs  in  bones  depends  on  the  age  of 
the  patient.  The  Tetracycline  rule  could  have  been  written  so  that  Instead  of  asking 
for  the  patient's  age,  the  program  would  have  asked,  "Are  the  patient's  bones  subject 
to  chelation?"  But  MYCIN  is  designed  for  use  by  non-experts  who  are  only  expected 
to  supply  primary  data  about  the  patient.  To  avoid  asking  about  chelation,  we  could 
have  the  program  Infer  this  condition  from  the  patient's  age. 


Thus,  we  could  have  written  MYCIN's  Tetracycline  rule  In  two  parts: 
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(1)  IF  the  patient's  bones  are  subject  to  drug  chelation, 

THEN  don’t  prescribe  Tetracycline. 

(2)  IF  the  patient  Is  less  than  8  years  old, 

THEN  the  patient’s  bones  are  subject  to  drug  chelation. 

However,  we  only  have  one  rule  to  determine  whether  chelation  might  occur  (2), 
so  it  was  convenient  to  collapse  this  argument  into  one  step:  if  the  patient  is  less 
than  8,  don't  prescribe  Tetracycline.  Imagine  the  presence  of  a  third  rule,  say: 

(3)  IF  the  patient  has  received  steroids  In  the  past  3  days, 

THEN  the  patient’s  bones  are  subject  to  chelation. 

Now  we  might  write 

(4)  IF  the  patient  Is  less  than  8  or 

has  received  steroids  In  the  past  3  days, 

THEN  don’t  prescribe  Tetracycline. 

And  we  have  again  "compiled  out"  the  Intermediate  concept  of  chelation. 

The  point  of  this  exercise  is  to  illustrate  that  there  are  many  ways  to  write  rules 
so  that  the  "right  answer"  (correct  medical  diagnosis  and  therapy)  comes  out.  In 
fact,  when  the  400  rules  were  written  for  MYCIN,  we  found  it  convenient  to  keep  the 
number  of  parameters  in  the  system  to  a  minimum,  making  it  a  practice  to  leave  out 
intermediate  concepts  (like  "chelation")  wherever  possible.  For  this  reason,  we  claim 
that  MYCIN's  rules  really  only  encode  the  input/output  behavior  of  medical  reasoning. 
Raw  data  feeds  into  the  rules,  raw  conclusions  come  out  (see  Section 
6 .2.2.2),  with  only  a  small  number  of  high-level  intermediate  concepts  tying  it 
all  together.  Consequently,  in  order  to  use  MYCIN's  rules  for  teaching,  we  must  either 
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add  more  knowledge  to  the  system,  or  rely  on  the  student  to  till  in  the  missing  steps, 
based  on  his  limited  classroom  and  clinical  experience. 

Since  it  seems  implausible  to  expect  a  second  year  medical  student  to  be  able 
to  figure  out  every  MYCIN  rule  on  his  own,  we  have  annotated  the  rules  with  fixed- 
text  (not  machine  readable)  justifications.  Moreover,  literature  references  cite 
published  articles  that  provide  further  information  (including  original  clinical  studies) 
for  illustrating  the  reasoning  behind  the  rule.1  Finally,  to  unveil  difficult  implementation 
problems,  comments  are  also  available,  though  they  are  usually  only  useful  to 
someone  with  a  thorough  knowledge  of  MYCIN.  At  best  they  bring  out  misgivings  that 
the  rule  author  has  about  the  adequacy  of  the  rule  for  covering  the  concept  he  is 
trying  to  incorporate  In  the  program.2  Examples  of  these  appear  in  Figure  6.2. 

Finally,  we  should  emphasize  that  the  problem  of  missing  reasoning  steps  is  not 
necessarily  solved  by  writing  intermediate  concepts  and  rules:  We  simply  don't  know 
offhand  which  concepts  to  Include.  Imagine  expanding  the  Tetracycline  rule  by 
introducing  three  intermediate  concepts:  chelation,  teeth  discoloration,  and 
undesirable  body  change: 


1  This  information  is  stored  in  a  hash  file  on  disk,  along  with  other  rule-specific 
annotations. 

2  For  example,  the  rule  may  be  a  gross  oversimplification,  using  one  or  two  crude 
measurements  to  check  for  a  patient  condition  that  would  require  dozens  of  finer 
considerations  for  precision.  Thus,  MYCIN  concludes  that  a  current  therapy  is  not 
effective  if  the  patient  has  not  "responded"  In  2  days.  To  do  this,  MYCIN  asks  the 
physician  whether  the  patient  has  responded—though  sometimes  it  takes  an  expert  to 
make  this  Judgement,  involving  examination  of  temperature  charts,  the  typical  course 
of  the  illness,  and  consideration  of  other  drugs  the  patient  Is  receiving.  (See  also  the 
comments  for  the  "Clostridia  rule"  In  Section  6.1.1.) 


Support  Knowledge  —  for  explaining  rules 


149 


youngster  *>  propensity  for  chelation  in  growing  bones 
Tetracycline  administration  &  propensity  for  chelation 
=>  teeth  discoloration 

teeth  discoloration  =  >  undesirable  body  change 
undesirable  body  changes  =  >  contraindication 


The  choice  of  steps  or  Intermediate  concepts  is  arbitrary,  of  course.  For 
example,  there  is  no  mention  of  how  the  chelation  occurs.  What  are  the  conditions? 
What  molecules  or  ions  are  involved?  Even  if  we  tried  to  fill  in  all  of  the  missing  steps 
in  MYCIN's  rules,  how  would  we  choose  the  right  level  of  abstraction  to  formalize  the 
reasoning  steps  involved?  If  our  purpose  is  good  performance,  the  tetracycline  rule  as 
first  stated  above  is  quite  adequate.  If  we  wish  to  explain  the  rule  to  a  student,  we'll 
need  some  mention  of  bone  and  teeth  involvement.1  But  if  the  purpose  is  to  develop  a 
system  that  can  truly  reason  about  Its  knowledge  (e.g.,  to  find  analogies  during  a 
knowledge  acquisition  or  learning  process),  all  the  links  that  a  physician  knows  might 
conceivably  be  useful. 

This  suggests  that  we  need  two  representations:  one  that  represents  compiled 
(or  optimized)  knowledge  (as  in  current  MYCIN  rules),  for  performance,  and  another 
representation  that  details  step-by-step  reasoning  from  empirical  data  to  final 
conclusions.  Ideally,  the  compiled  form  would  be  generated  from  the  detailed  version, 
and  for  exotic  cases  the  program  would  be  able  to  reason  on  the  detailed  level 
(careful  mode  In  the  style  of  Sussman's  program  writer),  looking  for  possible  bugs  in  its 
cookbook  (compiled)  knowledge.  Knowledge  acquisition  and  tutorial  programs  would 
very  likely  benefit  from  both  levels  of  representation. 

1  The  careful  reader  might  agree  that  teeth  discoloration  is  certainly  a  critical 
thing  to  know,  but  he  might  wonder  why  this  Is  satisfying.  Section  5.2.4 
characterizes  satisfying  explanations  in  terms  of  shared  underlying  models. 
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Does  a  MYCIN  rule  merely  leave  out  Invariable  steps  in  the  reasoning  chain? 
Can/should  we  view  rules  as  "looking  down  on"  a  network  of  some  kind  [Hart,  1975]? 
We  are  in  no  position  here  to  suggest  a  possible  representation  for  all  of  the 
physiological,  empirical,  and  social  knowledge  that  lies  behind  MYCIN's  rules.  It  is 
extensive,  that  much  is  certain,  and  for  the  moment  we  will  have  to  rely  to  a  large 
degree  on  the  justifications  and  literature  references.  Thus,  GUIDON  carries  on  a 
dialogue  about  the  USE  of  MYCIN's  rules,  and  cannot  clarify  the  meaning  of  the 
inferences  themselves.  To  see  how  the  situation  might  be  improved,  we  first  study  in 
more  detail  the  kinds  of  knowledge  that  MYCIN's  rules  ieave  out. 

5.2  2  Principles  Missing,  from  Optimized  Knowledge 

Examples  below  illustrate  three  kinds  of  principles  that  are  not  represented  in 
the  rules,  but  are  nevertheless  an  ini:  cate  part  of  the  support  knowledge  for 
understanding  them.  Data  abstraction  principles  are  concepts  tor  interpreting  case 
data:  Rules  tend  to  state  the  raw  data  directly.  Inference  principles  are  concepts 
for  interpreting  rule  conclusions:  Rules  tend  to  state  detailed  values,  without 
discriminating  trends.  Mechanistic  principles  are  concepts  that  underlie  the  rules 
and  serve  to  tie  premise  to  conclusion  (like  "chelation"). 

5. 2.2.1  Data  Abstraction  Principles 

The  first  type  of  optimization  we  will  consider  is  where  an  abstract  concept  for 
interpreting  case  data  has  been  replaced  in  a  rule  premise  by  a  specific  reference  to 
empiric  data.  Consider  "the  Leukopenia  rule"  (Rule  557). 
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RULE557 


If:  1)  The  Infection  which  requires  therapy  is  meningitis, 

2)  A:  A  smear  of  the  culture  was  not  examined,  or 

B:  Organisms  were  not  seen  on  the  stain  of  the  culture, 

3)  The  type  of  the  infection  is  bacterial, 

4)  The  results  of  a  recent  CBC  of  the  patient  are  available,  and 

5)  The  white  count  from  the  patient's  peripheral  CBC  (in  thousands) 
is  less  than  2.5 

Then:  There  Is  evidence  that  the  organisms  (other  than  those  seen  on 
cultures  or  smears)  which  might  be  causing  the  infection  is 
pseudomonas-aeruginosa  (.5)  e.coli  (.75)  klebsiel la-pneumoniae 
(.5) 

Figure  6.17,  Leaving  Out  Data  Abstraction  Principles:  The  Leukopenia  Rule 
In  terms  of  principles,  a  white  blood  count  of  less  than  2.5  would  be  considered  by  a 
physician  to  be  clear  evidence  of  "leukopenia. "  Rather  than  adding  a  new  parameter 
to  the  MYCiN  system  and  another  rule  of  the  form  "if  the  white  blood  count  is  less 
than  2.5  then  there  is  evidence  of  leukopenia,"  the  rule  author  "compiled"  this 
knowledge  into  the  rule. 


Many  other  examples  could  be  given  of  rules  in  MYCIN  that  skip  directly  from 
observation  (case  data)  to  long  lists  of  conclusions  without  explicit  representation  of 
intermediate  concepts  for  interpretation  of  the  data.  Part  of  the  problem  was  that 
intermediate  concepts  may  be  useful  for  organizational  constructs,  but  specific  rules 
demand  precise  consideration  of  empiric  details.  Catch-all  terminology  might  lead  to 
broad,  weak  rules  of  little  diagnostic  value. 

It  turns  out  that  in  general  a  white  blood  count  below  4.5  is  called  "leukopenia." 
[Harrison's,  1874]  Thus,  the  author  of  the  leukopenia  rule  evidently  wanted  evidence 
of  "pronounced  leukopenia."  It  is  somewhat  awkward  to  make  this  a  parameter  since  it 
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would  probably  have  no  use  elsewhere  in  the  system,  and  if  it  did,  we  wouldn't  be 
surprised  if  our  notion  of  "pronounced  leukopenia"  changed  in  different  contexts. 

Concept  broadening  is  a  more  general  problem,  however,  and  there  is  a  fairly 
straightforward  remedy.  Once  a  physician  included  the  pre-existing  MYCIN  concept 
"compromised  host"  in  a  new  rule  that  he  was  writing.  The  problem  is  that  he  meant 
something  more  specific  ("acute  leukemia  with  granulocytopenia  induced  by 
chemotherapy")  than  the  grounds  MYCIN  would  use  for  deducing  this  condition,  e  g., 
simply  the  fact  that  the  patient  is  undergoing  chemotherapy.  The  remedy  is  to  include 
both  the  general  concept  and  its  empiric  refinement  in  the  rule,  and  perhaps  annotate 
the  clauses  to  indicate  this. 

Applying  this  technique  to  the  Leukopenia  rule,  clause  4  could  then  be 
translated,  "the  WBC  is  less  than  2.5,  i.e.,  leukopenia."  However,  now  the  rule  set 
would  no  longer  be  optimized,  for  the  precondition  "leukopenia"  and  the  rule  that 

deduces  it  are  of  explanatory  benefit  only.1  In  Section  5.2.4  we  show  how  the 
explanatory  power  of  intermediate  concepts  like  "compromised  host"  and  "leukopenia" 
lies  in  their  relation  to  an  underlying  model  of  disease.  There  we  suggest  that  it  may 
be  impractical  to  structure  the  performance  knowledge  so  that  it  expresses  this 
model;  instead  we  may  need  to  maintain  separate  representations. 


1  Fagan  believes  that  rules  that  use  numerical  bounds  are  easier  to  understand 
if  they  refer  to  measurements  with  respect  to  the  norm,  e.g.,  "if  the  white  blood  count 
Is  HIGH."  He  uses  intermediate  rules  to  make  these  distinctions  in  different  patient 
situations  in  the  Ventilator  Management  program.  [Fagan,  at  al„  1970] 
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S.2.2.2  Inference  Principles 

The  second  characteristic  of  optimized  knowledge  is  where  inference  principles 
have  been  replaced  by  a  list  of  specific  conclusions.  To  illustrate  this,  we  return  to 
the  tabular  rule  shown  in  Figure  5.9.  While  tabular  rules  are  tremendously  useful  for 
acquiring  sets  of  implicitly  related  rules  from  experts,  the  representation  falls 
considerably  short  of  what  GUIDON  needs  for  conveying  the  principle  of  the  rule, 
imagine  that  you  are  a  student,  faced  with  the  table  of  parameter  ranges  and 
certainty  factors  shown  in  Figure  6.9.  How  are  you  going  to  remember  all  of  those 
numbers?  Wouldn't  it  be  better  to  know  the  principle  behind  that  table  instead?  At 
the  very  least,  we'd  be  better  off  if  we  had  a  handle  on  the  trends  or  most  significant 
conclusions  for  each  range.  Considerations  like  these  suggest  that  we  graph  the 
table  of  conclusions  to  see  if  there  is  a  pattern  we  can  easily  grasp  and  remember. 
The  table  for  the  CSF  glucose  rule  is  shown  in  Figure  6.18. 


Belief 


Figure  6.18.  Graph  of  the  Conclusions  Made  by  Rule  602  (Figure  6.9) 
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The  graphical  representation  brings  out  the  structure  of  the  table.  For  example, 
the  evidence  for  bacterial  meningitis  is  inversely  proportional  to  the  csf  glucose 
value;  on  the  other  hand,  a  glucose  value  below  16  tends  to  rule  out  viral  meningitis. 
Furthermore,  somewhere  between  5  and  10  an  Important  discrimination  can  be  made: 
above  1 0  there  is  no  way  to  differentiate  between  tb,  bacterial,  and  fungal  meningitis 
using  the  csf  glucose  value,  while  a  value  less  than  10  yields  no  evidence  for  tb  or 
fungal  but  strongly  supports  bacterial. 

To  recap,  using  this  chart  alone,  we  derived  principles  that  are  precisely  what 

we  want  to  teach  a  medical  student.1  But  notice  that  MYCfN  has  no  grasp  on  the 
concepts  "low  value"  or  "discrimination  point."  So  how  is  GUIDON  to  be  sure  that  the 
student  understands  what  is  essential  (worth  remembering)  about  this  rule?  Quite 
simply,  the  program  has  no  way  to  do  this  because  it  has  no  handle  on  the  principles 
involved. 

We  are  led  again  to  observe  that  MYCIN's  rules  constitute  "compiled" 
performance  knowledge.  (Think  of  the  table  as  the  "assembly  language"  form  of  the 
abstractions  we  have  stated  above.)  However,  our  observations  go  beyond  the 
problem  of  using  performance  knowledge  for  teaching,  imagine  the  difficulty  that  a 
trained  physician  would  have  in  trying  to  convince  himself  that  the  csf  glucose  rule  is 
correct:  he  will  have  to  go  through  the  very  same  analysis  we  produced  above!  When 
rules  implicitly  encode  principles  it  will  take  a  significant  amount  of  intelligence  to 
make  them  explicit  again.  Moreover,  there  is  the  danger  that  GUIDON  will  be  too  picky 


1  Of  course  students  will  use  different  techniques  for  committing  this  to  memory. 
For  example,  the  graph  has  a  strong  visual  effect  with  tb,  bacterial,  and  fungal 
marching  down  together. 
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about  numeric  ranges  and  long  lists  of  certainty  factors,  and  have  no  grasp  on 
higher  level  principles  it  should  be  trying  to  convey. 

We  have  considered  the  possibility  of  writing  a  program  that  could  analyze 
MYCIN's  tabular  rules,  using  general  heuristics  for  asking  questions  to  make  sense  of 
graphs.  However,  since  there  is  every  reason  to  believe  that  the  original  rule  author 
knew  the  principles,  we  would  be  better  off  having  him  represent  the  principles 
explicitly  in  the  first  place. 

6. 2.2.3  Mechanistic  Principles 

We  have  already  considered  how  MYCIN's  rules  lack  a  generalization  level  of 
data  abstraction  (premise)  and  Inference  (conclusions).  We  will  now  examine  an 
underlying  level  of  reasoning  that  ties  premise  and  conclusion  together.  This 
underlying  knowledge  Is  often  called  a  mechanistic  model. 

It  is  possible  that  the  medical  knowledge  we  are  trying  to  codify  Is  really  on  two 
levels  of  detail:  1)  principles  or  generalizations,  and  2)  empiric  details  or 
specializations.  Following  the  ideas  of  web  tutoring  [Norman,  Gentner,  &  Stevens, 
1976],  we  might  claim  that  a  student  should  be  taught  principles  at  first  to  set  up  a 
foundation  or  outline.  Then  when  the  student  begins  his  clinical  experience  as  an 
intern,  he  will  learn  to  cope  with  many  detailed  variations  of  these  principles.  The 
conflict  here  Is  that  principles  are  useful  for  getting  an  Initial  grasp  on  the  material,  as 
an  orientation  to  understand  kinds  of  data  and  Inferences,1  but  they  may  be 
inappropriate  for  practical  problem-solving. 

1  Assuming  that  the  principles  and  concrete  examples  are  presented  together— 
the  relationship  is  posited  to  be  what  provides  leverage  for  memory. 
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For  example,  "consider  the  portal  of  entry  of  the  organism  into  the  body,"  is  a 
principle  based  on  a  mechanistic  model  of  infection  that  is  taught  to  medical 
students,1  but  perhaps  it  is  a  poor  concept  to  use  in  a  performance  knowledge  base: 
rather  than  providing  diagnostic  clues,  "portal"  is  very  often  deduced  from  the 

diagnosis  itself.  The  following  excerpt  from  a  dialogue  with  a  physician  2  illustrates 
this  point: 


interviewer:  Okay....  What  do  you  think  about  "portal  of  entry"?  Is  that  of 
any  interest...? 

Physician:  ...if  this  is  an  uncomplicated,  if  this  case  is  what  it  appears 
to  be,  which  is  a  typical  case  of  pneumococcal  pneumonia,  there 
wouldn’t  be  any  question  in  my  mind  as  to  the  portal  of  entry... 

It's  obvious:  the  patient  has  pneumonia,  he  has  a  pocket  of  bacteria 
in  his  lungs  and  that's  obviously  the  way  the  bacteria  got  into  the 
blood. . . . 

interviewer:  ...lung  tissue... 

Physipian:  Uh-huh,  this  question  can  be  moot  if  the  patient  has  MORE  than 
one  site  of  Infection. 

Interviewer:  What  COULD  the  portal  of  entry  tell  you?  Dr.  Y  says  that  it’s 
retrospective,  it's  something  you  know  after  you’ve  already  decided 
what  the  infection  and  the  organisms  are,  and  that  doesn't  help  you 
conclude  the  identity  of  the  organism  or  the  name  of  the  Infection. 

Physician:  In  this  particular  case  it  doesn't  add  anything.  If  you  were 
dealing  with  a  a  case  of,  well,  consider  this  case,  young  20  year 
old  patient  has  his  teeth...  Say  he  has  rotten  teeth,  and  he  goes  to 
a  dentist  and  then  2  or  3  weeks  later  develops  bacterial 
endocarditis,  temperature  of  105  degrees,  very  ill,  and  you  have  a 
coccus  growing  out  of  the  blood.  Then,  portal  of  entry  adds 
something.  Because  the  typical  organism  that  causes  endocarditis 
under  those  circumstances  is  strep-virldans.  Strep-virldans  is 
normal  flora  In  the  mouth  and  a  typical  history  of  strep-vir idans 
bacterial  endocarditis  is  that  of  having  dental  work  done.  And  so 
the  fact  that  the  portal  of  entry  was  probably  the  mouth  DOES 
influence  your  decision  about  what  the  organism  Is. 

interviewer:  So  It’s  the  OTHER  history  that  told  you  the  probable  portal  of 
entry. . . ? 


1  "With  relatively  minor  variations,  the  development  of  an  infectious  disease 
follows  a  consistent  pattern.  The  parasites  enter  the  body  through  the  skin, 
nasopharynx,...  or  other  portal,  and  a  regular  sequence  ensues...."  [Harrison's,  1974], 
p.  724.  Compare  with  the  model  we  formalize  in  Section  6.2. 4.3. 

2  The  interviewer  was  simulating  GUIDON  and  the  physician  was  playing  the  role 
of  a  student.  Thus,  the  physician  was  given  only  limited  information  about  the  case 
and  needed  to  request  more  as  necessary.  One  purpose  of  the  interview  was  to 
study  how  the  accumulating  evidence  changed  the  physician's  understanding  of  the 
case. 
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Physician:  Uh-huh,  and  that  [hypothetical]  case  is  different  from  this  one 
in  that  you  don't  have  an  infection  in  the  mouth  [corresponding  to 
the  infection  in  the  lungs].  [In  the  hypothetical  case]... all  you 
have  is  a  history  of  work  having  been  done  in  the  mouth... 

Interviewer:  Right.  I  see.  Things  having  to  do  with  wound  infection  or 
something  like  this,  is  that  useful  sometimes...? 

Physician:  Ah...  There  are,  I'm  sure  there... 

Interviewer:  Well,  I’m  probably  getting  into  things  I  don’t  want  to  get  into 
right  now. . . 

Physician:  I'm  sure  that  it  might  be  useful... 

Interviewer:  ...trauma  wounds,  it  seems  like... 

Physician:  ...sure,  sure,  that  might  be  useful. 

interviewer:  ...it's  used _  Okay,  so  let's  start  to  wrap  this  up  here. 

Physician:  You  may  not  know  what  the  portal  of  entry  is  I  think  typically. 

If  you  have  several  Infectious  sites  in  the  body,  you  have  a  trauma 
wound  on  the  hand,  you  have  meningitis,  a  kidney  Infection,  and 
those  are  the  kind  of  cases  that  HYCIN  will  be  asked  to  consult  on. 

I  don’t  see  that  the  physician  has  any  way  of  knowing.  And  I  think 
that  what  Dr.  Y  means  by  retrospective  decision  Is  that  if  there’ s 
agreement  between  the  bacteria  growing  in  the  blood  and  bacteria 
from  one  of  those  sites  of  infection,  then  you  assume  that  the 
portal  of  entry  was  from  that  site  of  infection. 


From  this  perspective,  "portal"  is  a  concept  that  a  physician  most  often  uses 
after  the  fact  to  form  a  complete  picture  that  satisfies  his  model  of  the  infectious 
disease  process.  However,  consideration  of  “portal"  only  infrequently  provides 
leverage  for  problem  solving  itself.  This  suggests  that  it  may  be  difficult  or  impossible 
to  expect  a  single  set  of  MYCIN  diagnostic  rules  to  serve  both  as  concise,  "clincher" 
methods  for  efficiently  getting  to  the  right  data,  and  still  represent  a  model  of 
disease.  Put  another  way,  a  student  may  need  the  model  if  he  is  to  understand  new 
associations  between  disease  and  manifestations,  but  he  will  be  an  inefficient  problem 
solver  if  he  always  attempts  to  directly  convert  that  model  to  a  subgoal  structure  for 
solving  ordinary  problems.  More  generally,  the  knowledge  we  use  to  justify  what  we 
know  may  be  used  only  tacitly  to  solve  the  problem  by  providing  feedback  (meta¬ 
knowledge)  that  we  are  getting  it  right,  that  everything  fits  [Barr,  1979]. 
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If  we  attempt  to  make  our  model  of  disease  explicit  in  MYCIN's  rules,  we  may 
find  that  the  organizing  structure  gets  in  the  way,  as  one  physician  has  claimed  about 
the  parameter  "portal  of  entry."  It  shouldn't  be  expected  that  general  principles  will 
be  a  good  mold  for  constructing  interpretations  o!  every  case.  According  to  this  view, 
principles  are  good  for  summarizing  arguments,  and  good  to  fall  back  on  when  you've 
lost  grasp  on  the  problem,  but  they  don't  drive  the  process  of  expert  reasoning. 

Perhaps  one  good  way  to  get  a  handle  on  this  problem  is  to  consider  how 
MYCIN's  purely  goal-directed  approach  to  diagnosis  Is  entirely  inconsistent  with  data- 
directed  thinking.  Indeed,  when  we  analyze  the  different  approaches  used  in  MYCIN’s 
rule  set,  we  notice  that  there  are  kinds  of  subtrees:  (1)  deep,  clear  subgoal  structure 
that  reflects  an  approach  for  doing  diagnosis,  and  (2)  a  shallow,  broad  layer  of  rules 
for  triggering  on  specific  observations.  To  a  large  degree,  this  difference  reflects  the 
differing  styles  of  rule  authors.  One  group  tended  to  make  concepts  explicit,  as  can 
be  seen  in  Figure  5.1  A,  illustrating  the  basic  subgoal  structure  for  tackling  a  problem. 
The  other  group  generally  preferred  to  express  their  knowledge  in  "cookbook"  form, 
going  directly  from  observation  to  diagnosis  instead  of  lumping  observations  into 
intermediate  principles.  This  second  group  argued  that  they  didn't  reason  from 
principles  and  it  was  difficult  to  approach  a  problem  in  this  way.  They  even  claimed 
that  they  would  have  "failed"  if  they  attempted  to  break  every  problem  down  in  a 
strict  subgoal  fashion  that  used  explanatory  principles  like  "portal  of  entry"  and 
"compromised  host." 

However,  it  is  possible  that  the  two  groups  were  dealing  with  different  kinds  of 
problems.  The  group  that  used  intermediate  concepts  and  formalized  a  subgoal 
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structure  of  some  depth  were  dealing  with  hard  data  (for  identifying  an  unknown 
organism  given  observed  culture  data).  Here  a  few  basic  factors  enable  one  to  write 
solid  rules  that  quickly  latch  onto  the  answer.  (Indeed  the  rules  for  these  intermediate 
concepts  tend  to  be  mutually  exclusive.)  When  soft,  clinical  concepts  like 
"compromised  host"  are  considered,  they  don't  play  a  central  role  in  decision  making. 

On  the  other  hand,  the  other  group  of  rule  writers,  those  that  designed  a  shallow 
subgoal  structure  with  many  rules  for  a  general  goal,  may  have  been  confronted  with  a 
more  nebulous  kind  of  pattern-matching  problem-,  determining  an  unknown  organism 
from  clinical  data  alone.  There  are  many  pieces  of  data  available,  but  few  hard  and 
fast  rules  to  relate  them.  The  underlying  theory  recedes  to  the  background,  and  the 
expert  tends  to  approach  his  problems  simply  in  terms  of  associations  between 
observed  data  and  bottom-line  conclusions.  This  may  have  promoted  a  rule  writing 
style  that  discouraged  introducing  intermediate  concepts,  even  where  they  might 
have  been  appropriate,  e.g.,  the  concept  of  "leukopenia"  described  above. 

5.2.3  Implications  for  Tutoring 

The  analysis  above  is  partly  speculation  to  account  for  the  missing  principles  we 
observe  in  the  MYCIN  rule  set.  However,  our  immediate  concern  is  to  consider  the 
implications  of  tutoring  from  this  rule  set.  To  the  extent  that  MYCIN's  rules  are 
"empiric,"  that  is,  capture  every  practical  consideration,  they  will  be  too  detailed  for 
conveying  principles  to  students.  As  described  above,  MYCIN's  rules  are  too  detailed 
in  three  ways:  1)  they  check  for  many  specialized  cases  without  lumping  the 
observations  Into  general  concepts  (such  as  "high"  and  "low"  values);  2)  they 
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conclude  a  uniform  list  of  possibilities  that  fails  to  distinguish  between  rare  and 
expected  outcomes  (indicating  inference  discrimination  points);  and  3)  they  omit 
underlying  abstract  (mechanistic)  concepts  (like  "impairment  of  host'defense 
mechanism").  Besides  requiring  the  student  to  look  for  patterns  in  the  rules  to  find 
the  principles,  detailed  knowledge  of  this  kind  will  inhibit  GUIDON'S  ability  to  convey 
principles  by  appropriately  selecting  cases  and  weighing  presentations. 

A  likely  result  is  that  GUIDON  will  attempt  to  teach  the  wrong  level  of  knowledge. 
It  might  demand  support  for  hypotheses  on  the  level  of  detailed  data,  when  the 
student  may  most  need  to  know  and  remember  underlying  concepts  and  relations  that 
medical  science  has  abstracted  from  the  observed  data.  Using  MYCIN's  rules  alone, 
GUIDON  might  possibly  teach  how  to  conceptually  subdivide  a  problem,  but  it  could 
never  provide  explanatory  principles  that  justify  that  subdivision. 

5.2.4  The  Use  of  Models  for  Explanation 

The  point  of  this  subsection  is  very  simple,  though  the  exposition  is  long.  We 
claim  that  satisfying  explanations  of  MYCIN-like  rules  are  couched  in  terms  of  models 
that  experts  use  to  convince  themselves  that  there  is  sufficient  evidence  for  some 
condition.  Specifically,  explanations  of  rules  enable  one  to  conclude  that  the  critical 
events  of  a  process  have  taken  place,  and  so  an  end  condition  claimed  by  the  rule  is 
true.  Not  all  critical  events  of  the  script  are  mentioned  in  the  explanation  or 
supported  directly  by  the  rule.  Many  are  implicit  and  derived  from  default  knowledge 
about  the  world. 


« 


Support  Knowledge  --for  explaining  rules 


161 


The  basic  observation  is  that  to  write  a  set  of  judgmental  rules  to  determine 
some  parameter  (a  proposition  that  postulates  evidence  for  some  condition),  you  must 
have  some  notion  of  what  constitutes  sufficient  evidence  for  that  condition. 
Moreover,  each  time  that  you  write  such  a  rule,  you  instantiate  that  notion  of 
sufficient  evidence  by  your  choice  of  premise  clauses  in  the  rule  and  the 
hypothesizes  that  you  claim  these  clauses  support.  Evidence  for  a  process  is  given  in 
terms  of  evidence  for  critical  events  or  states  of  the  process.  Rules  that  conclude 
that  a  process  has  taken  place  (or  that  a  condition  exists)  mention  factors  that  allow 
one  to  infer  that  the  critical  events  have  taken  place  or  that  the  critical  conditions 
hold.  If  you  know  the  sequence  of  critical  events,  that  is,  the  process  model,  and  can 
infer  that  each  has  been  enabled  in  a  particular  case,  then  you  have  evidence  that 
the  end  condition  has  taken  place. 

Each  rule  is  an  argument  that  the  end  condition  has  taken  place  (with  a 
particular  certainty).  If  the  process  model  is  known  to  the  student,  then  explaining  a 
rule  involves  arguing  that  the  premise  provides  evidence  that  the  end  condition  holds. 
Default  knowledge  may  be  necessary,  for  MYCIN  rules  tend  to  refer  to  only  one 
condition  in  the  process.1  In  accord  with  this  view  of  what  a  rule  is,  the  rule  set  is  a 
kind  of  mapping  that  relates  observable  phenomena  in  the  world  to  the  model  of  a 
process.  New  rules  or  rule  modifications  are  designed  to  complete  the  coverage  of 
observable  factors  to  critical  events  that  enable  the  process.  For  example,  following 
the  infectious  disease  model  we  present  in  this  section,  we  could  imagine  that  a 


1  Probably  because  of  the  assumption  that  case  factors  are  independent,  so 
they  can  be  considered  in  separate  rules-*leading  to  weak  rules  with  small  certainty 
factors. 
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research  physician  will  look  for  new  rules  to  relate  patient  occupations  to  infection, 
knowing  that  exposure  to  an  organism  is  a  critical  event  in  the  infection  process.  It  is 
this  connection  between  an  occupation  and  the  notion  of  exposure  that  is  not 
represented  in  MYCIN's  rules. 

To  make  our  point  about  the  use  of  models  for  explaining  rules,  we  have  chosen 
to  justify  MYCIN's  40  rules  for  diagnosing  the  cause  of  meningitis  (roughly  75%  of  the 
system's  rules  that  are  specific  to  meningitis).  The  classifications  presented  below 
were  derived  by  the  author  by  examining  these  40  rules;  preliminary  results  were 
improved  with  the  help  of  physicians.  Development  of  the  model  proceeded  in  three 
phases;  1 )  determining  the  key  factors  of  rules;  2)  classifying  the  key  factors  by 
kind  of  evidence;  and  3)  abstracting,  from  the  kinds  of  evidence,  a  model  of  what 
constituted  sufficient  evidence  of  infection. 

6. 2.4.1  The  Structure  of  Rules;  Determining  Key  Factors 

Analysis  of  the  40  rules  showed  that  many  of  them  had  certain  clauses  in 
common;  from  this  we  defined  the  rule  schemas  that  are  described  in  Section  5.1.4. 
These  clauses  set  the  context  of  the  rule,  e.g.,  "bacterial  meningitis  with  no  lab  data 
available."  Of  the  remaining  clauses,  some  we  classified  most  as  restrictions,  and  the 
one  or  two  that  remained  constituted  the  key  factor(s)  of  the  rule.  The  "petechial 
lesions"  rule  (rule  541)  exemplifies  this  classification. 
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RULE541 


CONTEXT, 

1)  The  Infection  which  requires  therapy  is  meningitis, 

2)  A:  A  smear  of  the  culture  was  not  examined,  or 

B:  Organisms  were  not  seen  on  the  stain  of  the  culture, 

3)  The  type  of  the  infection  is  bacterial, 

KEY-FACTOR, 

4)  Petechial  is  one  of  the  types  of  rash  which  the  patient  has, 
RESTRICTIONS, 

5)  Purpuric  is  not  one  of  the  types  of  rash  which  the  patient  has, 

6)  Leukemia  is  not  one  of  the  diagnoses  of  the  patient, 

7)  Lymphoma  is  not  one  of  the  diagnoses  of  the  patient,  and 

8)  The  patient  has  not  received  cytotoxic  drugs  just  prior  to  or 

during  this  infection 

CONCLUSION, 

There  is  weakly  suggestive  evidence  (.3)  that  Neisseria- 

meningitidis  is  one  of  the  organisms  (other  than  those  seen  on 
cultures  or  smears)  which  might  be  causing  the  infection 


Figure  6.19.  Classification  of  Clauses  in  the  "petechial  lesions"  Rule 
A  heuristic  for  detecting  restriction  clauses  is  that  they  are  stated  negatively,  e  g., 
clauses  6-8  above.  We  have  identified  seven  kinds  of  restriction  clauses  in  MYCIN’s 
rules: 


1)  Data  Screen  (a  parameter  that  indicates  whether  a  class 
of  data  is  .available,  e.g.,  CSF  results  are  available  iff  a  lumbar 
•  puncture  was  performed) 


2)  World  Knowledge  Relation  (relationships  between  age  or 
sex  and  other  factors,  e.g.,  alcoholism  is  not  considered  in  children) 


3)  Refinement  (further  specification  of  a  factor,  e.g., 
"neurosurgery"  further  specifies  "surgery") 


4)  Other  Cause  (evidence  that  may  account  for  the  key 
factor,  and  so  invalidate  this  particular  rule,  e.g.,  leukemia. 
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lymphoma,  and  cytotoxic  may  also  cause  a  petechial  rash,  making  it 
wrong  to  conclude  that  Neisseria  is  necessarily  the  cause  (Figure 
5.19)) 


5)  Default-overruling  (evidence  that  invalidates  default 
conclusions,  e.g.,  a  head  injury  overrules  the  relevance  of  the 
patient's  age) 


6)  Distinguished  Subset  (condition  that,  when  it  occurs  in 
combination  with  the  key  factor,  must  be  treated  as  a  special  case, 
e  g.,  hospital-acquired  infections  for  new  borns,  purpuric  rash  in 
combination  with  petechial  rash) 


7)  Evidence  Tightening  (additional,  optional  factor  that  shores 
up  the  key  factor  argument,  e.g.,  Neisseria  is  seldom  the  cause  of 
hospital-acquired  meningitis,  and  the  absence  of  an  epidemic  is 
double  evidence  that  clinches  the  argument) 


These  restriction  clauses  are  lumped  together  into  three  kinds  of  annotations  in 
the  current  version  of  GUIDON:  screening  clauses,  exceptions,  and  refinements.  Using 
this  knowledge,  GUIDON  can  subtract  off  the  context  and  restrictions  to  get  at  the 
key  factor.  Once  the  pivotal  or  key  factor  has  been  identified,  it  remains  to  determine 
how  this  is  evidence  for  the  conclusion,  e.g.,  what  does  a  petechial  lesion  have  to  do 
with  Neisseria? 


6. 2.4.2  Classifying  the  Parameters  by  Evidence  Source 

Without  prior  medical  knowledge,  it  was  Impossible  to  determine  how  the  key 
factor  was  specifically  related  to  the  conclusion.  However,  It  was  possible  to  find 
regularity  by  classifying  the  parameters  by  the  source  of  evidence.  We  discovered 
that  the  35  clinical  parameters  in  the  40  rules  could  be  broken  into  three  categories. 
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corresponding  to  (1)  direct  observations  of  disease,  (2)  history  of  the  present  illness, 
and  (3)  predisposing  factors  for  disease.  The  final  category  is  further  broken  into  (a) 
exogenous  factors  that  affect  the  body  and  (b)  endogenous  characteristics  of  the 
body  that  make  it  prone  to  disease.  This  classification  is  shown  in  detail  in  Figure 
5.20.  (Parameters  appear  in  all  caps;  definitions  are  given  in  Appendix  C  for  the 
interested  reader.) 

By  examining  the  kinds  of  evidence  used  in  the  rules,  we  were  a  step  closer  to 
understanding  the  concept  "clinical  evidence  of  disease"  that  these  rules  deduce. 
However,  this  classification  was  insufficient,  for  it  seemed  to  be  orthogonal  to  the 
underlying  understanding  we  were  groping  for.  For  example,  a  congenital  malformation, 
an  endogenous  factor  (III.B.  I.b),  obviously  must  be  a  predisposition  of  some  kind — the 
same  reason  for  considering  alcoholism,  an  exogenous  factor  (III.A.2.b).  What  role  do 
these  conditions  play  that  makes  their  effect  analogous?  What  is  the  mechanism? 
The  remarks  labelled  "intuition"  represent  our  first  attempt  to  relate  the  source  of 
evidence  to  a  description  of  th8  disease  process. 
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I .  Indirect  Observations  of  Present  Causative  Organism/Disease 

[  Intuition .  existing  disease  may  have  a  visible  effect  ] 

A.  Laboratory  Tests 

CBC  WBC  TBRISK  (biopsy,  ppd,  chest-xray  scar)  CXRAB 

B.  Signs 

OCNERVE  RASHES  SKININFECT 

C.  Symptoms 

MUMPSYM 


II.  History  of  Present  Illness 

[Intuition,  recent  diagnoses  are  evidence  for  current  problems ] 
CLINEV-TUBERCULOSIS  OTITIS-MEDIA  EPIGLOTTITIS 


III.  Predisposing  Factors 


A.  Exogenous 

[Intuition:  factors  that  have  changedlchallenged  the  body  ] 

1.  Environment 

[Intuition:  exposure  is  a  prerequisite  for  some  diseases  ] 

ANIMALEXP  CROWD  EPIDEHIC  NOSOCOMIAL  TBRISK  (contact) 

2.  Body  Infractions 

[Intuition:  entry  of  organism  or  weakening  of  defenses 
leads  to  disease  ] 
fi)  Procedures 

CNSRADIATE  CARDIACSURGERY  GU-MANIP 
NEUROSURGERY  SPLENECTOMY 

b)  Drugs 

ALCOHOLIC  CYTOTOXIC  JUNKIE  STEROIDS 

c)  Traumatic  Wounds 

BURNED  HEADTRAUMA  HEADINJDEF 


B.  Endogenous 

[  Intuition:  what  the  patient " brought "  to  this  situation  ] 

1.  Physical  Characteristics 

[ Intuition :  some  diseases  are  correlated  with  body  type  ] 

a)  Demographic 

AGE  RACE 

b)  Congenital 

CNSMALFORM 

c)  Acquired 

2.  Past  Medical  History 

[Intuition:  previous  diseases  change  the  body  and/or  persist  ] 
DIABETIC  LYMPHOMA  LEUKEMIA  SICKLECELL 

Figure  6.20.  Preliminary  Classification  of  COVERFOR  Factors 
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5. 2.4.3  Critical  Events:  Constructing  a  Model  of  Sufficient  Evidence 

Proceeding  from  the  classification  of  evidence  source,  we  formed  a  model  of 
sufficient  evidence  for  disease— what  you  need  to  know  before  you  believe  that  some 
organism  is  causing  disease.  It  Is  a  description  of  the  mechanism  of  infection, 
including  mention  of  an  agent  and  critical  events  that  must  take  place  for  infection  to 
occur.  Thus,  we  have  a  representation  of  the  COVERFOR  concept--"organism  (other 
than  those  seen  on  cultures  or  smears)  that  might  be  causing  the  infection."  The 
critical  events  1  of  the  description  are  derived  directly  from  our  shared  scientific 
model  of  the  infectious  disease  process,  which  generally  describes  infection  in  terms 
of  war:  an  invading  organism,  port(al)  of  entry,  protective  white  ceils  that  fight 
infection,  and  so  on. 

The  mechanism  description  sets  constraints  on  the  rules  that  will  conclude 
COVERFOR  by  characterizing  the  kind  of  evidence  that  they  must  mention  in  their 
preconditions.  The  frame-like  representation  of  this  description  is  shown  below. 
Particular  parameters  can  refer  to  one  or  more  slots:  we  say  that  they  satisfy  the 
slots.  Explaining  a  rule  consists  of  fitting  the  rule  premise  and  action  to  the  frame  of 
the  parameter  it  determines.  That  is,  demonstrating  that  the  rule  satisfies  all  slots  of 
the  description. 


1  Our  use  of  the  term  "critical  events"  comes  from  Stevens  [Stevens  &  Collins, 
1976]  who  used  to  term  to  relate  scripts  used  In  WHY  to  models  of  meteorological 
processes. 
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COVERFOR-FRAME 


BUG-AGENT:  (FARM  "implies  Infection  by"  ORGS) 

BUG-ORIGIN:  ( PARM  "implies  that  the  patient  has  been 
exposed  to"  ORGS) 

PORTAL-PATHWAY:  (PARM  "allows  access  of  organisms  from" 
SITE1  "to"  SITE2) 

IMMUNOLOGY:  (PARM  "implies  impairment  of  the  host-defense 
mechanism" ) 

DIAGNOSIS-CAUSE:  (ORG  "causes"  PARM) 


Figure  5.21.  Mechanism  Description  for  the  Parameter  COVERFOR 


The  COVERFOR  mechanism  description  tells  the  story  of  an  organism  (BUG- 
AGENT)  that  is  normally  found  at  BUG-ORIGIN;  it  enters  the  body  and/or  proceeds  to 
the  infection  site  via  PORTAL-PATHWAY,  where  conditions  favor  its  growth 
(IMMUNOLOGY);  thus,  an  infection  is  visible  by  certain  symptoms  and  signs 
(DIAGNOSIS-CAUSE).1  The  factors  given  in  Figure  6.20  are  related  to  the  frame  by 
associating  a  property  with  each  factor  that  instantiates  a  slot  of  the  frame.  The 
general  reorganization  Is  shown  below: 

slot  factor  classifications 


BUG-ORIGIN:  ENVIRONMENT 

PORTAL-PATHWAY:  INVASIVE-PROCS,  TRAUMATIC -WOUNDS 

IMMUNOLOGY:  DRUGS,  PAST-MED-HISTORY,  PHYSICAL-CHARACTERISTICS 

DIAGNOSIS -CAUSE:  OBSERVATIONS,  PRESENT-ILLNESS 

Figure  6.22.  Association  of  COVERFOR  Factors  with  Frame  Slots 


1  This  story  suggests  that  we  call  these  frames  scripts  [Schank  &  Abelson, 
1976]  [Stevens  8  Collins,  1978].  However,  it  is  too  soon  to  say  whether  frames  for 
other  parameters  (e.g.,  CONTAMINANT,  SIGNIFICANCE  etc.)  will  have  the  same  story¬ 
like  character.  For  now,  we  will  simply  observe  that  the  slots  are  a  more-or-less 
ordered  set  of  conditions. 
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Figure  6.23  illustrates  how  the  COVERFOR  frame  can  be  used  to  "explain"  a  rule. 
For  comparison,  a  physician's  explanation  for  the  same  rule  appears  in  Figure  6.24. 


RULE535 


The  fact  that  the  patient  Is  an  alcoholic  allows  access  of  organisms 
from  normal  flora  of  throat  and  mouth  to  lungs  (by  reaspiration  of 
secretions) . 

Figure  5.23.  GUIDON'S  Explanation  of  the  "alcoholic  rule"  (Figure  5.6) 


Physician:  I'd  be  interested  knowing  if...  if  this  patient  were  an  alcoholic 
would  be  one  of  the  things. 

Interviewer:  Why  would  that  be  useful  to  you? 

Physician:  Alcoholics,  particularly  skid-row  bums,  typically  pneumonia  is  an 
occupational  hazard  of  being  a  skid-row  bum.  They  drink  themselves 
Into  a  stupor  and  then  aspirate  their  own  oral  secretions  and 
develop  pneumonia.  Also,  there  have  been  some  studies  to  show  that 
alcohol  inhibits  the  cilllary  motion  of  the  respiratory  tree  and 
that  would  prevent  clearing  of  aspirated  bacteria,  which  would 
predispose  to  pneumonia. 

Interviewer:  In  particular.  It's  the  pneumonia  that  makes  you  curious  about 
that? 

Physician:  A  59  year  old  male  with  pneumonia  would  make  me  wonder  what  the 
predisposing  cause  was.  Pneumonia  is  not  a  terribly  uncommon 
disease,  but  when  it  appears  you  wonder  about  the  etiology--the 
conditions  that  set  this  particular  59  year  old  man  up  to  get 
pneumonia.  In  fact,  one  of  the  things  you’d  do  in  following  this 
patient  after  the  patient  is  discharged,  you  will  seek  in  3  weeks  to 
determine  whether  this  infiltrate  in  the  chest  x-ray  has  cleared. 

And  if  it's  not.  It'll  raise  the  question  of  some  underlying  disease 
process  causing  the  pneumonia. 

Figure  6.24.  Physician's  Search  for  Predisposing  Factors 


5.2.4.4  Recap:  The  Process  of  Explaining  a  Rule 

To  back  up  a  bit  to  see  what  we  have  accomplished  and  what  remains  to  be 
done,  recall  that  the  original  problem  of  this  section  was  to  explain  additional  medical 
knowledge  would  enable  GUIDON  to  explain  MYCIN's  rules,  that  Is,  to  convince  a 
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student  that  a  rule  makes  sense.  Even  if  we  had  a  huge  semantic  net  about 
physiological  processes,  how  would  we  select  from  it  to  explain  a  rule?  What 
relationships,  what  level  of  detail,  would  provide  satisfying  explanations? 

The  breakthrough  in  this  problem  comes  from  viewing  the  rule  as  statement  that 
has  to  be  proved  (albeit  informally),  so  that  the  problem  of  explaining  a  rule  is 
tantamount  to  providing  sufficient  evidence  that  the  conclusion  holds.  What 
constitutes  sufficient  evidence  is  obviously  dependent  on  the  particular  conclusions 
of  the  rule  (e.g.,  diplococcus-pneumoniae  with  strong  certainty),  but  more  importantly, 
our  notion  of  sufficient  evidence  for  the  rule  is  strongly  grounded  in  the  concept  we  are 
trying  to  construct.  Thus,  we  are  led  to  the  observation  that  a  rule's  preconditions 
must  satisfy  the  implicit,  abstract  "preconditions"  of  the  concept  mentioned  in  the 
rule's  action— a  rule  is  something  like  a  sketch  of  an  existence  proof. 

An  explanation  for  a  rule  has  two  constraints  to  satisfy:  1 )  it  must  account  for 
all  preconditions  and  rule  conclusions,  and  2)  it  must  demonstrate,  through  plausible 
reasoning  (involving  a  model  of  the  listener's  knowledge),  that  there  is  sufficient 
evidence  for  the  process  condition  claimed  by  the  rule's  conclusion  (e.g.,  "X  is  an 
organism  thought  (on  the  basis  of  clinical  evidence  alone)  to  be  causing  an  infection'*). 
The  algorithm  used  by  GUIDON  to  explain  a  rule  Is  summarized  below. 
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Input :  Rule  premise  and  action 

Annotations:  rule  schema,  screening  relations,  exception 
and  refinement  clauses 
Frame  for  the  action  parameter 

Description  of  precondition  parameters  In  terms  of  the  frame's  slots 

Step  1:  Remove  contextual  clauses  using  the  rule  schema  template 
2:  Remove  screening,  exception,  and  refinement  clauses 
3:  Output:  Translations  of  all  frame  slots  Instantiated  by  remaining 
parameters 

4:  Output:  List  of  clauses  that  have  not  been  accounted  for 
Figure  5.26,  GUIDON'*  Algorithm  for  Explaining  a  Rule 

Thus,  to  produce  the  explanation  for  the  alcoholic  rule,  GUIDON  (1)  removed  the 
(contextual)  clauses  that  are  implicitly  part  of  the  COVERFOR  frame  (bacterial 
meningitis  diagnosed  on  the  basis  of  clinical  evidence);  (2)  removed  the  (screening) 
clause  about  the  age  of  the  patient;  and  (3)  fetched  the  PORTAL-PATHWAY  property 
of  ALCOHOLIC  and  used  It  to  fill  in  the  corresponding  slot  In  the  COVERFOR  frame.  The 
algorithm  currently  used  in  GUIDON  does  not  attempt  to  fill  in  every  slot  of  the  frame, 
for  this  would  require  additional  knowledge  about  the  organisms  and  considerably  more 
sophisticated  reasoning.  For  example,  to  satisfy  the  BUG-ORIGIN  slot  for  the  alcoholic 
rule,  the  program  would  have  to  use  default  knowledge  about  the  BUG-AGENT 
Diplococcus-pneumoniae  (mentioned  in  the  rule  conclusion)  to  account  for  its  presence 
In  the  mouth.1  GUIDON'S  explanations  therefore  only  partly  satisfy  requirement  "1" 
mentioned  above:  They  account  for  every  clause  of  the  rule,  but  leave  it  up  to  the 
student  to  fill  in  the  BUG-AGENT  slot  with  the  organisms  mentioned  in  the  rule 
conclusion,  and  to  fill  In  other  slots  in  the  frame  using  default  knowledge. 


1  In  fact,  Diplococcus  normally  exists  there,  even  In  healthy  patients. 
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Producing  explanations  from  a  model  of  sufficient  evidence  is  interesting 
because  a  satisfying  explanation  for  any  one  of  40  rules  is  couched  in  terms  of  a 
concise  description  of  a  single  underlying  process.  For  GUIDON'S  explanations  to 
work,  the  understander  must  know  the  mechanism  description,  and  proceed  from  it, 
using  the  slots  as  expectations  or  necessary  components  of  the  argument.  Given 
fitting  instantiations  for  each  of  the  slots,  he  will  find  that  the  inference  is  satisfying , 
that  is,  it  fits  his  model  of  what  constitutes  evidence  for  the  given  concept.  Of 
course,  in  developing  the  program  further,  we  will  want  to  perform  experiments  to 
determine  the  validity  of  this  theory  of  explanation. 

6.2.4.G  Completeness  of  the  Explanation 

As  might  be  expected,  GUIDON'S  algorithm  for  explaining  a  rule  is  strongly  limited 
by  the  domain  knowledge  available  to  the  program.  Indeed,  the  major  reason  for 
Implementing  this  primitive  form  of  explanation  routine  was  not  to  produce  a  practical 
system  that  explained  all  rules,  but  rather  to  illustrate  some  of  the  problems  involved 
in  making  sense  of  a  rule.  GUIDON  cannot  really  fit  a  rule  to  the  COVERFOR  frame 

because  It  lacks  default  knowledge  about  organisms  and  body  sites.1  For  example,  if  a 
key  factor  refers  to  very  contagious  organisms,  GUIDON  has  no  way  of  checking  to 
see  if  the  organisms  in  the  rule's  conclusion  actually  are  very  contagious. 

Nevertheless,  GUIDON  manages  quite  well  to  get  at  the  essence  of  the 
explanation  because,  first,  It  has  the  necessary  knowledge  to  strip  away  contextual 
and  context-restricting  Information,  and  second,  rules  are  optimized  so  that  the 


1  See  [Reither,  1076]  for  a  study  of  reasoning  with  default  knowledge. 
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parameters  that  remain  in  the  rule's  premise  are  those  that  really  clinch  the  argument: 
they  enable  the  frame,  and  default  knowledge  makes  everything  else  fit  into  place. 
8y  characterizing  the  enabling  parameter(s)  in  terms  of  the  underlying  model,  GUIDON  is 
supplying  the  essential  step  that  makes  everything  else  go  forward.  To  see  this 
again,  consider  the  "crowd  rule"  (rule533)  and  the  program's  explanation. 

RULE533 


If:  1)  The  Infection  which  requires  therapy  Is  meningitis, 

2)  A:  A  smear  of  the  culture  was  not  examined,  or 

B:  Organisms  were  not  seen  on  the  stain  of  the  culture, 

3)  The  type  of  the  infection  Is  bacterial,  and 

4)  The  patient  does  live  In  a  crowded  environment 

Then:  There  Is  weakly  suggestive  evidence  (.2)  that  Neisseria- 

meningitidis  Is  one  of  the  organisms  (other  than  those  seen  on 
cultures  or  smears)  which  might  be  causing  the  infection 


RULE533 


The  fact  that  the  patient  does  live  in  a  crowded  environment  implies 
that  the  patient  has  been  exposed  to  very  contagious  organisms. 


Figure  6.26.  The  "Crowd  Rule”  and  its  Explanation 
GUIDON'S  explanation  is  again  superficial  and  it  involves  no  real  reasoning,  but  it 
makes  a  connection  on  the  level  that  counts:  Living  In  a  crowded  environment  is 
relevant  because  of  the  exposure  to  organisms.  That  Neisseria  is  contagious  is  default 
knowledge  that  completes  the  argument  that  therapy  for  the  Infected  patient  should 
cover  for  this  organism. 


Some  of  the  slots  in  the  COVERFOR  frame  appear  to  be  very  general.  What 
reason  do  we  have  to  believe  that  GUIDON  won’t  be  called  on  to  explain  them  further? 
The  explanation  for  the  "CNS  radiation  therapy  rule”  (rule516)  states,  "The  fact  that 
the  patient  has  undergone  recent  radiation  therapy  to  the  CNS  implies  impairment  of 
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the  host-defense  mechanism."  Well,  what  Is  the  impairment?  The  conclusion  of  the 
rule  mentions  the  default  list  of  "unusual  bacteria"  for  meningitis  (staphylococcus, 
.  e.coli,  klebsieila,  and  pseudomonas)  that  we  see  in  rules  that  mention  Leukemia, 
Lymphoma,  etc.,  so  the  impairment  seems  to  be  generally  non-specific  (see  the  sickle 
cell  rule  below  for  comparison),  indeed,  when  we  consult  an  expert  we  find  that  the 
mechanism  is  not  well-understood.  Ironically,  the  best  reason  to  believe  that 
GUIDON'S  explanations  (of  rules  like  the  CNS  radiation  rule)  will  be  acceptable  is  that 
this  is  the  best  medicine  can  do.  Connection  to  the  underlying  model  must  be  enough, 
for  it  has  been  sufficient  to  teach  rules  like  this  to  many  doctors  over  many  years. 

The  explanation  of  the  "sickle  celt  disease"  rule  is  similar  to  that  for  the  "CNS 
radiation  therapy"  rule,  but  here  medical  science  enables  us  to  delve  into 
physiological  details.  The  sickle  cell  disease  rule  states  that  "if  the  race  of  the 
patient  is  black  and  the  patient  has  had  sickle  cell  disease,  then  this  is  weak 
evidence  that  diplococcus-pneumoniae  is  one  of  the  organisms  that  might  be  causing 
the  infection." 

Consider  how  a  student  who  knew  the  COVERFOR  frame  might  proceed  to  explain 
this  rule.  Assuming  that  he  knew  that  only  Blacks  get  sickle  cell  disease  (thus 
consideration  of  the  race  of  patient  Is  a  screening  clause),  how  would  he  make  sense 
of  this  rule?  There  anpear  to  be  two  good  possibilities:  either  sickle  cell  disease 
causes  diplococcus-pneumoniae  infection  (by  enabling  PORTAL-PATHWAY  or 
IMMUNOLOGY)  or  sickle  cell  disease  is  caused  by  diplococcus  (DIAGNOSIS-CAUSE). 
But  he  probably  knows  that  sickle  cell  disease  is  a  genetic  condition  and  so  it 
develops  "by  Itself."  Besides,  he  will  have  trouble  explaining  why  diplococcus  doesn't 
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cause  sickle  cell  disease  in  other  races,  requiring  yet  another  causal  factor.  The 
reasoning  involved  here  bears  some  resemblance  to  that  investigated  by  Stevens  and 
Collins  in  the  domain  of  meteorology  (in  particular,  for  understanding  the  cause  of 
heavy  rainfall)  [Stevens,  Collins,  &  Goldin,  1978], 

GUIDON'S  explanation  of  this  rule  is:  "The  fact  that  the  patient  has  sickle  cell 
disease  implies  impairment  of  the  host-defense  mechanism."  As  before,  GUIDON  offers 
no  details  about  the  nature  of  host-defense  impairment.  However,  in  this  case  more  is 
known.  The  impairment  Is  caused  by  angular  blood  cells  (sickle  cells)  that  clog  the 
vessels  of  the  spleen  and  kill  it.  The  (empirically  observed)  function  of  the  spleen  is 
to  act  as  a  filter  to  trap  diplococcus.  This  function  is  impaired,  so  the  IMMUNOLOGY 
condition  is  enabled.  These  details  are  interesting  and  insightful,  and  they  place 
medicine  on  a  stronger  scientific  footing,  perhaps  providing  flexibility  fo»  m  *  , 
about  difficult  cases  where  expectations  are  violated.  We  will  not  consider  further 
how  these  details  could  be  represented  in  GUIDON;  though  we  observe  that  the 
explanation  given  above  (using  general  notions  like  filtration  and  function-impairment) 
suggests  that  the  idea  of  using  an  explanation  model  might  be  extended  to 
characterize  sufficient  evidence  for  a  condition  in  the  COVERFOR  frame,  just  as  the 
frame  we  have  given  characterizes  evidence  of  the  COVERFOR  concept.1 

6. 2.4. 6  More  Sophisticated  Reesoning 

Finally,  It  Is  Important  to  point  out  that  the  process  of  fitting  the  COVERFOR 


1  Work  reported  in  [Rieger  &  Grinberg,  1977]  and  [de  Kleer,  1979]  might  be 
relevant  here. 
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frame  may  involve  much  more  sophisticated  reasoning  than  simply  filling  in  slots.  In 
particular,  when  an  argument  Is  clinched  by  intermediate  reasoning  steps  based  on 
two  or  more  of  the  rule's  preconditions,  it  will  be  necessary  to  relate  the  preconditions 
to  show  how  they  together  "fill  in"  a  slot.  This  problem  is  illustrated  by  the 
"nosocomial  rule"  (rule  661). 

RULE561 


If:  1)  The  infection  which  requires  therapy  is  meningitis,  and 

2)  The  infection  was  acquired  while  the  patient  was  hospitalized,  and 

3)  Meningococcal-disease  is  not  one  of  the  epidemic  diseases  to  which 
the  patient  has  been  exposed  recently 

Then:  There  is  suggestive  evidence  (.5)  that  neisser ia-meningitidis  is 
not  one  of  the  organisms  (other  than  those  seen  on  cultures  or 
smears)  which  might  be  causing  the  Infection 

Figure  5.27.  The  Nosocomial  Rule 


Notice  that  this  rule  is  saying  that  the  COVERFOR  frame  is  not  satisfied.  It  does 
this  by  showing  that  for  a  given  BUG-AGENT  (Neisseria),  the  BUG-ORIGIN  slot  is  not 
satisfied.  We  know  from  our  general  medical  facts  (what  we  have  been  calling  default 
knowledge  here)  that  Neisseria  is  seldom  the  cause  of  bacterial  meningitis  that 
originates  during  a  hospital  stay.  This  alone  is  evidence  against  Neisseria,  however 
the  rule  author  included  another  clause  (3)  to  shore  up  the  argument. 


Suppose  that  the  patient  was  admitted  with  Neisseria  meningitis,  but  it  was 
undiagnosed  at  the  time.  When  the  bacterial  meningitis  is  detected  during  his  hospital 
stay,  it  will  be  termed  "nosocomial."  Concluding  that  it  could  not  be  Neisseria 
(because  it  is  nosocomial)  would  be  wrong.  To  minimize  misinterpretations  like  this,  we 
check  to  see  if  there  is  a  Neisseria  epidemic,  if  not,  then  this  is  reasonable  evidence 
that  the  patient  did  not  tnttr  the  hospital  with  an  undiagnosed  case  of  Neisseria. 
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Thus,  the  rule  author  has  hit  BUG-ORIGIN  from  both  sides:  If  the  infection  originated  in 
the  hospital,  it  is  not  Neisseria,  because  that  is  unusual,  and  if  it  originated  outside 
the  hospital,  that  too  is  unlikely  to  be  Neisseria-caused  because  there  is  no  epidemic 
(Neisseria  is  very  contagious,  another  general  fact). 

Since  we  have  evidence  that  Neisseria  cannot  satisfy  BUG-ORIGIN,  we  conclude 
that  it  does  not  satisfy  the  COVERFOR  frame  (-.5).  This  illustrates  that  a  rule 
explanation  program  must  have  the  capability  to  form  a  chain  of  reasoning  from  the 
preconditions.  However,  note  that  the  existence  of  the  frame  will  be  a  significant 
advantage  for  focusing  search.  That  is,  the  program  needs  to  prove  (or  disprove,  if 
evidence  is  negative)  that  the  conditions  of  the  frame  are  satisfied.  Thus  the  model 
of  sufficient  evidence  serves  to  reduce  the  combinatorics  of  the  problem. 

Finally,  some  COVERFOR  rules  can  be  characterized  as  being  statistical, 
grounded  on  purely  empiric  observations,  e.g.,  the  age  rule  (Figure  6.13).  They  are 
default  inferences  that  say,  "given  no  other  information  about  the  patient,  this  is  what 
typically  causes  an  infection."  The  explanation  model  is  not  satisfied  by  rules  like  this, 
and  it  appears  that  whatever  representation  we  use  for  the  COVERFOR  parameter,  it 
will  need  a  special  slot  to  account  for  default  evidence.  This  is  not  surprising,  for 
several  recent  knowledge  representation  languages,  e  g.,  K.RL  [Bobrow  &  Winograd, 
1977]  and  UNITS  [Stefik,  1979],  have  chosen  to  label  or  distinguish  inferences  in 
this  way. 
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5.2.4. 7  What  We  Accomplished 

To  recap,  we  began  by  asking  questions  about  MYCIN's  rules,  trying  to  account 
for  all  of  the  clauses.  "Does  diplococcus  CAUSE  sickle  cell  disease?  Or  does  sickle 
cell  disease  leave  one  SUSCEPTIBLE  to  infection  with  diplococcus-pneumoniae?"  Off 
hand,  we  couldn't  tell  which  way  the  causality  went  in  these  rules.  Though,  when  we 
step  back,  we  see  that  these  questions  about  causality  and  susceptibility  are  asked 
by  an  understander  with  an  underlying  model  of  the  kinds  of  relations  he  expects  to 
see  mentioned  in  the  rules  e  g.,  organisms  cause  a  diagnosed  disease,  previous 
illnesses  weaken  the  body.  In  seeking  an  explanation,  the  understander  tries  to  fit 
this  rule  to  his  model  of  disease.  Put  another  way,  if  a  human  teacher  (or  program)  is 
to  provide  satisfying  explanations,  he  will  have  to  respect  this  model  as  a  constraint 
that  the  understander  is  trying  to  satisfy.  And  if  the  student  is  a  complete  novice,  it 
will  be  important  to  sketch  out  the  model  itself  so  that  the  student  will  be  able  to 
learn  (accept  as  meaningful)  specific  relations  like  this. 

To  illustrate  how  such  a  model  could  be  used  by  a  program  to  explain  the  rules, 
we  formulated  a  frame  to  account  for  about  40  of  MYCIN's  rules.  We  believe  that  the 
algorithm  provides  satisfying  explanations  for  about  half  of  these  rules.  It  falls  short 
where  deeper  explanations  are  possible  (as  for  sickle  cell  disease)  or  where  clauses 
must  be  related  to  one  another  in  order  to  fill  in  a  slot  of  the  frame  (as  for  the 
nosocomial  rule).  We  began  by  classifying  the  parameters  because  it  appeared  that 
there  were  different  principles  or  means  for  deducing  a  parameter.  We  then 
discovered  that  each  parameter  (and  rule)  was  referring  (explicitly)  to  just  part  of  an 
aggrega'e  of  conditions,  and  that  explaining  the  rule  involved  filling  in  unmentioned 
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principles  from  default  knowledge.  (Thus,  one  rule  mentioned  evidence  lor  entry  of  an 
organism  into  the  body,  another  mentioned  an  immunological  weakness,  etc.) 
Explanation  of  a  particular  rule  involves  satisfying  a  single  concept  that  is  shared  by 
all  rules  of  the  set,  namely  the  concept  of  what  constitutes  sufficient  evidence  for 
the  parameter  being  concluded  about.  This  underlying  single  concept  is  represented 
here  as  a  frame  and  is  called  the  explanation  model  for  the  parameter. 


5.2.4.S  Use  of  Explanation  Models  In  Knowledge  Acquisition 


Restatement  of  rules  via  explanation  models  provides  a  simple,  rudimentary 
facility  for  making  semantic  checks  of  the  knowledge  base.  This  is  important  because 
it  ensures  correctness  of  newly  entered  rules.  Moreover,  by  having  the  tutor  give  its 
own  explanation  of  a  rule,  it  gives  the  rule  author  an  opportunity  to  enter  more 
annotations  for  students  and  other  experts  who  will  need  help  to  understand  the  rule. 
Figure  5.28  illustrates  both  uses. 

RULES!! 


The  fact  that  the  patient  has  had  neurosurgery  allows  access  of 
organisms  from  air  and  skin  normal  flora  to  cerebral-spinal  fluid. 

Can't  explain  the  clause: 

the  time  since  the  neurosurgical  operation  was  performed  is  less  than  2 
months 


RULE513 


The  fact  that  the  patient  has  had  neurosurgery  allows  access  of 
organisms  from  air  and  skin  normal  flora  to  cerebral-spinal  fluid. 

Can't  explain  the  clause: 

1)  A  smear  of  the  culture  was  examined,  or 
2)  Organisms  were  seen  on  the  stain  of  the  culture 

Figure  5.28.  Two  Rules  GUIDON  Cannot  Explain 
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Rule  511  illustrates  how  GUIDON  is  unable  to  account  for  the  importance  of 
knowing  how  long  ago  neurosurgery  occurred.  In  fact,  if  the  program  knew  this,  it 
would  be  able  to  justify  the  table  of  conclusions  that  appear  in  the  rule's  action  (see 
Section  5.1.3  for  related  discussion).  By  showing  this  flawed  explanation  to  the  rule 
author,  a  GUIDON- augmented  knowledge  acquisition  program  would  be  able  to  focus  his 
attention  on  the  clause  that  needs  to  be  annotated  in  the  rule  justification. 

Rule  513  illustrates  an  actual  case  where  explanation  models  were  used  to 
detect  an  error  in  the  knowledge  base.  In  the  process  of  testing  GUIDON,  the  author 
used  the  explanation-generation  function  on  all  of  the  COVERFOR  rules  (40  pairs). 
Rule  513  is  properly  labelled  as  being  an  example  of  the  schema  COVERFOR.CLINICAL, 
but  the  OR  clause  is  wrong  (both  subclauses  should  be  negated).  This  error  had  been 
undetected  for  nearly  2  years.  MYCIN's  current  capability  to  do  syntactic  type¬ 
checking  is  helpful  for  catching  rule  errors,  but  this  example  illustrates  that  using 
even  a  simple  analytic  approach  (can  we  account  for  all  of  the  clauses?)  might  be 
more  effective. 

6.2.5  Conclusions  about  Support  Knowledge 

We  conclude  here  that  knowledge  representation  for  tutoring  should  distinguish 
among  levels  of  abstraction.  Problem-solving  knowledge  that  uses  a  minimum  number 
of  intermediate  reasoning  steps  (involving  multiple  levels  of  abstractions  of  case  data, 
hypotheses  and  relations  among  them)  might  be  perfectly  adequate  for  high 
performance,  but  will  be  insufficient  for  explanation  to  users  and  students  who  lack 
the  concepts  for  gluing  it  all  together.  Understanding  the  "tetracycline  rule"  involves 
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knowing  about  chelation,  drug  migration  In  the  body,  and  even  social  criteria  for  what 
constitutes  undesirable  facial  appearance.  An  Important  observation  to  make  here  is 
that  the  meaning  of  a  rule  can  never  be  completely  specified  [Wittgenstein,  1958}. 
We  need  only  prepare  to  give  reasonable  explanations.  At  some  point,  even  a  human 
tutor  would  give  up,  concluding  that  the  student  knew  so  little  about  the  world  that 
there  was  no  hope  to  reason  further  on  the  level  of  diagnosing  and  treating  infectious 
diseases. 

Many  of  our  other  observations  about  the  structure  of  MYCIN's  knowledge  led  to 
the  same  conclusion,  and  many  of  our  observations  may  be  useful  for  designers  of  new 
representations.  However,  it  is  not  the  purpose  of  this  thesis  to  suggest  a  new 
representation,  but  only  to  characterize  knowledge  for  tutoring  in  terms  of  a  critique 
of  the  existing  MYCIN  rule  base.  We  want  to  emphasize  that  our  critique  is  part  of  an 
ongoing  struggle  in  the  MYCIN  team  to  decide  how  to  best  use  rules  for  representing 
knowledge;  it  is  by  no  means  a  sweeping  condemnation  of  the  formalism  itself. 
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5.3  Strategical  Knowledge  —  for  planning  solutions 

In  previous  sections,  we  have  examined  the  use  of  structural  knowledge  to 
organize  rules  and  parameters,  and  support  knowledge  to  justify  rules,  in  this  section 
we  relate  the  AND/OR  tree  of  MYCIN-like  rules  and  subgoals  to  a  deep  structure  for 
working  out  a  problem  [Brown,  Collins,  &  Harris,  1977]  (hereafter  this  paper  will  be 
referred  to  as  AILS).  We  characterize  the  observed  behavior  of  questions  asked  by  a 
consultant  and  an  "output"  diagnosis  as  a  surface  structure  of  the  problem  solution. 
We  argue  that  current  EMYCIN  knowledge  bases  lack  planning  knowledge  for  ordering 
methods  (rules)  into  a  line  of  reasoning,  and  that  this  knowledge  is  conceptually 
equivalent  to  the  meta-rule  formalism  implemented  by  Davis  [Davis,  1976].  Finally, 
we  demonstrate  that  addition  of  this  planning  knowledge  in  the  form  of  strategies  for 
controlling  the  search  (construction)  of  the  AND/OR  tree  would  provide  a  basis  for  1) 
providing  assistance  to  complete  partial  student  solutions;  2)  planning  which  rules  and 
topics  should  be  included  in  the  lesson  plan;  3)  summarizing  the  use  of  rules  to  attain 
a  goal  (motivate  why  the  rules  were  considered  at  a  given  point  in  the  problem 
solution);  and  4)  modeling  approaches  taken  by  the  student  to  solve  the  problem  and 
interpreting  his  hypotheses. 

This  analysis  is  based  on  the  prescriptive  statement  made  in  AILS: 


Without  explicit  awareness  of  the  largely  tacit  planning 
and  strategic  knowledge  inherent  in  each  domain,  it  is  difficult 
for  a  person  to  "make  sense  of"  many  sequences  of  behavior 
as  described  by  a  story,  a  set  of  instructions,  a  problem 
solution,  a  complex  system,  etc....  The  teacher  should 
articulate  for  that  domain  the  higher-order  planning  knowledge 
and  strategic  knowledge  for  formulating  and  revising 
hypotheses  about  what  something  means. 
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The  hypothesis  here  is  that  providing  a  person  with  the  means  to  understand  the 
steps  of  a  problem  solution,  is  providing  him  with  the  basis  for  applying  his  problem¬ 
solving  methods  to  other,  similar  problems. 

6.3.1  Surface  and  Deep  Structure  of  Rule  Systems 

As  mentioned  above,  a  person  trying  to  understand  a  MYCIN-like  consultation 
observes  that  pieces  of  case  data  are  requested  by  the  program  (Figure  5.29). 
These  questions  are  conceptually  terminals  hanging  below  an  AND  (rule)  node  in  the 
subgoal  tree  (one  portion  corresponding  to  Figure  5.29  is  shown  in  Figure  6.30. 1  In  the 
terminology  of  the  AILS  paper,  a  rule  node  is  a  method  for  achieving  a  goal  (e.g., 
“organisms  that  might  be  causing  the  infection")  by  setting  up  "deltacts"  or  some  set 
of  subgoals  to  achieve  (the  premise  of  a  particular  rule).  Thus  the  tree  of  rules  and 
subgoals  constitutes  a  deep  structure  trace  that  Brown,  et  al.  postulate  is 
constructed  when  the  understander  makes  sense  of  the  surface  problem  solution.2 
(See  also  [Greeno,  1976].) 


1  Generally  AND/OR  trees  for  the  infectious  disease  rule  base  are  3-5  levels 
deep;  this  is  one  reason  that  we  believe  a  goal-directed  dialogue  does  not  become 
unmanageably  hard  to  follow  (the  stack  of  goals  is  easy  to  keep  in  mind). 

2  Note  that  in  AILS,  subgoals  follow  from  subgoals  in  a  way  that  is  not  possible  in 
MYCIN--intervening  methods  appear  to  be  missing.  For  example,  in  the  story  about  how 
a  beggar  obtained  soup  from  a  rich  household,  the  subgoal  of  "getting  inside  the 
house"  Is  followed  directly  by  the  subgoal  of  "conning  the  maid  for  permission  to  go 
inside."  From  our  perspective,  the  beggar  is  applying  the  method  by  which  one  gets 
inside  by  getting  a  servant  to  let  him  in.  (An  alternative  method  would  be  to  break  in 
unseen.)  The  MYCIN  rule  formalism  provides  an  interesting  framework  for  viewing  the 
understanding  process  outlined  in  AIL8. 
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31)  Has  Pt538  ever  undergone  any  type  of  surgery? 

**  YES 

32)  Does  Pt538  have  a  history  of  NEUROSURGERY? 

**  NO 

33)  Does  Pt538  live  In  a  crowded  environment? 

**  NO 

34)  Do  you  suspect  recent  alcoholic  history  In  Pt538? 

**  NO 

35)  Is  meningitis  a  hospital-acquired  Infection? 

**  YES 

36)  Is  Pt538^s  clinical  history  consistent  with  EPIGLOTTITIS? 
**  NO 

37)  Is  PtSSS^s  clinical  history  consistent  with  OTITIS-MEDIA? 
**  NO 

38)  Has  Pt538  ever  undergone  splenectomy? 

**  NO 

39)  Is  Pt538  a  burn  patient? 

*•  YES 


Figure  6.20.  MYCIN  Consultation  Excerpt  Corresponding  to  the  Protocol 


GOAL 


COVERFOR 


HYPOTHESIS  METHOD 

-  E.COLI  - (Rule511  )- 

-  NEISSERIA  — (Ru1e533)- 

{( Ru  1  e535  )- 

(Rule559)- 

p(Rule545)- 
-  H. INFLU  — j 

L(Rule395) 


•DELTACT* 

032  NEUROSURGERY 
Q33  CROWD 
Q34  ALCOHOLIC 

Q38  SPLENECTOMY 

Q35  NOSOCOMIAL 

036  EPIGLOTTITIS 
037  OTITIS-MEDIA 


I —  PSEUOOMO.  —(Rule 5 78)—  Q39  BURN 


Figure  6.30.  Portion  of  the  AND/OR  tree  Corresponding  to  the 
Questions  Shown  in  Figure  6.29  (reorganized  according 
to  the  hypothesis  each  rule  supports). 
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As  discussed  in  AILS,  it  is  not  sufficient  for  a  student  to  know  ail  of  the  possible 
methods  he  can  bring  to  bear  on  a  problem.  For  example,  a  student  who  knows  what 
kinds  of  algebraic  transformations  can  be  used  to  solve  for  *x"  In  "x**2  -8*1"  could 
only  proceed  to  apply  the  methods  randomly  if  he  didn't  have  a  plan  for  solving  the 
problem  (i.e.,  have  schemas  for  kinds  of  problems  that  can  be  tackled  using  different 
approaches  or  lines  of  reasoning.)  A  plan  sets  up  a  sequence  of  applications  of  methods 
that  might  get  you  closer  to  the  solution  (though  this  is  not  guaranteed).  Plans  can  be 
thought  of  as  rules  of  thumb  or  strategies  for  dealing  with  recognizable  situations.  It 
is  this  planning  knowledge  that  AILS  suggests  we  might  convey  directly  to  the 
student. 

6.3.2  Representing  Planning  Knowledge  in  Meta-rules 

Davis  showed  how  domain  rules  for  achieving  a  goal  might  be  ordered  and  pruned 
by  meta-rules.  These  meta-rules  are  invoked  just  before  the  object-level  rules  are 
applied  to  achieve  a  goal.  An  example  of  an  infectious  disease  meta-rule  is  shown 

below.  Observe  that  this  is  a  strategy  for  pursuing  a  goal.1  In  a  simple  way,  it  sets  up 
a  line  of  reasoning;  in  particular,  it  provides  motivation  for  pursuing  some  subgoals 
before  others:  it  will  order  the  questions  asked  by  the  system,  and  hence  change  the 
surface  structure  of  the  consultation. 


1  In  particular,  this  meta-rule  might  be  associated  with  the  goal  "identity  of  the 
organism."  It  will  be  invoked  to  order  the  rules  for  every  subgoal  in  the  search  tree 
below  this  goal.  (Meta-ruleOOl  is  based  on  the  observation  that  one  of  the  group  of 
organisms  known  as  the  enterobacteriaceae  is  more  likely  to  be  causing  pelvic- 
abscess  infection  than  a  grampos-rod  organism.) 
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META-RULE001 


If:  1)  the  Infection  Is  pelvic-abscess,  and 

2)  there  are  rules  which  mention  In  their  premise 
enterobacterlaceae,  and 

3)  there  are  rules  which  mention  In  their  premise 
grampos-rods. 

Then:  There  Is  suggestive  evidence  (.4)  that  the  former 
should  be  done  before  the  latter. 

Figure  6.31.  A  MYCIN  Planning  Rule 


In  Figure  6.29  no  meta-rule  has  ordered  questions  31-39.  Each  question  roughly 
corresponds  to  application  of  a  single  (deep  structure)  method,  and  these  methods 
are  applied  randomly  for  each  goal.  Any  strategical  explanation  of  why  MYCIN  asked 
question  32  before  question  33  will  be  spurious:  the  problem-solving  system  is  simply 
applying  rules  (methods)  exhaustively. 


From  our  perspective  of  tutoring,  meta-rules  are  an  important  part  of  an  expert's 
explanation  for  applying  a  particular  solution  method  at  a  particular  time.  This  is 
consistent  with  Davis'  computationally-motivated  use  of  meta-rules  to  reduce  the 
combinatorics  of  searching  a  large  knowledge  base. 

Consider  the  problem  of  teaching  rules  to  a  student.  We  postulate  that  he  will 
find  it  easier  to  remember  the  rules  for  a  given  goal  if  he  has  some  strategy  for 
applying  them,  i.e.,  some  indexing  scheme  for  ordering  their  retrieval.  In  particular,  he 
might  recognize  the  situation  described  by  meta-ruleOOl  above  and  follow  that 
strategy  for  recalling  particular  rules  to  consider  next.  Similarly,  there  might  be 
domain-independent  strategies  for  examining  the  evidence  gathered  so  far  and 
seeking  to  confirm  it,  prove  it  inadequate,  or  further  split  the  hypothesis  space. 
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The  GUIDON  system  does  not  currently  use  domain  meta-rules  in  its  tutorials. 
However,  as  discussed  in  Section  8.2.3. 2,  there  are  tutorial  rules  that  are 
used  to  convey  domain-independent  diagnostic  strategies  to  the  student  at  the  time 
he  requests  assistance  for  working  out  a  subgoal. 

6.3.3  Meta-strategies 

It  has  been  observed  that  teaching  strategies  explicitly  is  not  always  effective 
[Wescourt,  et  at.,  1978].  For  example,  it  is  not  always  evident  how  Polya's  problem¬ 
solving  strategies  should  be  applied  to  particular  problems.  During  diagnosis,  it  is 
quite  possible  that  one  strategy  will  be  useful  for  collecting  the  initial  data  and 
focusing  the  search,  then  another  strategy  will  come  into  play  to  do  a  depth-first 
confirmation  of  the  most  likely  hypothesis,  and  other  strategies  will  determine  when  to 
consider  revising  a  hypothesis.  Burton  8>  Brown  have  done  pioneering  work  in 
incorporating  knowledge  of  a  student's  strategies  in  the  student  model,  and  have 
considered  some  of  the  problems  of  detecting  changes  in  strategy  [Burton,  1979]. 
The  use  of  meta-strategies  for  diagnosis  has  not  been  explored  in  GUIDON. 

6.4  Epilogue:  What  is  Meta-knowledge? 

References  to  "meta-knowledge"  have  become  increasingly  frequent  in  this 
decade  by  educators,  psychologists  and  Al  researchers  [Davis  &  Buchanan,  1977] 
[Barr,  1979]  [Collins,  et  al.,  1975]  [Collins,  1978]  [Flavell,  1977].  However  it  is 
not  clear  that  there  is  a  single  definition  that  encompasses  all  of  the  ways  in  which 
the  word  has  been  used.  In  fact,  there  appear  to  be  three  distinct  kinds  of 
knowledge  that  we  want  to  label  meta-knowledge  (Figure  5.32). 
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Type  1:  Knowledge  about  the  extent  of  what  you  know 

Type  2:  Meta-levels  of  knowledge:  structure,  support  &  strategy 
(domain-specific  knowledge  about  other  knowledge) 

Type  3:  Knowledge  about  the  thinking  process 

Figure  5.32.  Kinds  of  Meta-knowledge 

One  meaning  of  meta-knowledge,  used  by  cognitive  psychologists,  is  knowledge 
about  the  extent  of  other  knowledge.  For  example,  your  ability  to  indicate  (more  or 
less  accurately)  whether  you  could  say  the  16  points  of  the  compass  without  being 
reminded  of  them  first  is  knowledge  about  the  extent  of  what  you  know.  MYCIN's 
ability  to  "read  the  rules"  gives  it  the  capability  to  say  what  it  knows.  Rule  models  or 
program-generated  patterns  abstracted  from  the  rules  are  another  example.  Thus 
meta-knowledge  of  this  kind  is  knowledge  that  can  be  determined  by  introspection.  It 
has  the  sense  of  describing  what  is  known  in  general  terms. 

On  the  other  hand,  the  meta-knowledge  that  we  considered  in  this  chapter-- 
knowledge  that  structures,  supports,  and  controls  the  use  of  performance  rules--is 
additional  domain-specific  knowledge  that  happens  to  be  about  other  knowledge.  It  is 
tempting  to  call  these  other  "levels"  of  knowledge  in  the  sense  that  strategy  rules, 
for  example,  look  down  upon  and  control  the  use  of  problem-solving  methods,  and 
support  knowledge  provides  detailed  mechanisms  and  facts  that  underlie  and  hence 
justify  methods.  Thus,  one  level  of  knowledge  either  depends  on  or  refers  to  another, 
leading  to  the  term  "meta-/m/  knowledge"  or  simply  "meta-knowledge."  The  rule 
schemas,  explanation  models,  and  meta-rules  described  in  this  dissertation  are  of  this 
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type.  Annotations  in  recent  frame-oriented  languages  (FRL  [Goldstein  &  Roberts, 
1977];  KRL  [Bobrow  &  Winograd,  1977];  UNITS  [Stefik,  1979])  that  explicitly 
characterize  the  way  properties  are  inherited  in  a  semantic  structure  are  also 
examples  of  meta-level  knowledge  [Barr,  1979]. 

The  third  use  of  the  term  "meta-knowledge"  is  to  characterize  knowledge  about 
the  thinking  process  itself.  Flavell's  "metamemory"  research  illustrates  this.  As  one 
aspect  of  his  research,  Flavell  has  found  that  people  learn  to  make  judgments  about 
the  difficulty  of  retrieving  certain  information  from  memory,  e  g.,  it  is  easier  to  recall  a 
list  of  16  words  in  the  series  "north,"  "north-northeast,”  .  .  .  north-northwest"  than  a 
set  of  10  randomly  selected  words  [Flavell  &  Wellman,  1977].  This  is  knowledge 
about  the  capabilities  of  memory.’  This  form  of  meta-knowledge  differs  from  those 
described  above  in  that  it  does  not  refer  to  other  things  that  we  know  about,  but  is 
instead  knowledge  about  our  ability  to  know  or  learn. 

Finally,  we  should  point  out  that  complex  judgments  can  use  a  mixture  of  these 
kinds  of  meta-knowledge.  Indeed,  we  can  see  all  three  forms  in  a  protocol  analyzed 
by  Collins  [Collins,  1978].  A  subject  was  asked,  "Is  the  Nile  longer  than  the  Mekong 
River?"  Figure  5.33  relates  Collins'  analysis  of  the  subject's  underlying  inferences  to 
the  forms  of  meta-knowledge  described  above.1 2 


1  Interestingly  enough,  Flavell  has  evidence  that  this  knowledge  is  learned  and 
might  be  important  for  problem-solving  effectiveness. 

2  Collins  attributes  inference  2  to  the  subject's  knowledge  of  other  long  rivers 
("meta-induction  from  cases").  However,  note  that  this  inference  also  depends  on 
the  subject's  perception  of  his  memory  process;  At  this  moment  the  subject  does  not 
recall  that  the  Mekong  is  long  (1)  and  he  knows  that  waiting  will  not  change  that 
belief--hence,  he  "would  know  it  right  now"  if  it  were  so  (2). 
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Subject's  inference 


Form  of  Meta-knowledge 


1.  I  don’t  know  that  the  Mekong  is  Type  1:  extent  of  knowledge 
extremely  long. 


2.  I  would  know  that  the  Mekong  Is  Type  3:  knowledge  about  memory 

extremely  long  if  it  were.  process 

(Therefore,  probably  the  Mekong  is 
not  extremely  long.) 


3.  The  importance  of  a  river  depends 
in  part  on  length. 


Type  2:  abstraction  (meta-level 
description  of  facts ) 


4.  The  Mekong  is  not  very  important  Type  2:  importance  of  knowledge 

( meta-level  relation) 

(Therefore,  probably  the  Mekong  is 
not  extremely  long.) 


Figure  5.33.  Meta-knowiedge  Used  in  Plausible  Reasoning  (after  Collins) 


Our  experience  with  the  MYCIN  program  suggests  that  these  kinds  of  meta¬ 
knowledge  might  in  fact  be  intricately  interwoven.  As  stated  above,  MYCiN  knows 
what  it  knows  about  infectious  diseases  (meta-knowledge  type  1 )  because  it  can 
read  its  rules.  However,  MYCIN's  knowledge  of  rule  syntax  that  enables  it  to  read 
rules  (templates  of  rule  predicates)  is  the  basis  for  this  capability.  From  the  human 
programmer's  viewpoint,  templates  are  part  of  the  grammar  of  rules,  and  so  in  a  sense 
constitute  the  underlying  knowledge  for  decoding  them.  With  respect  to  the 
representation  of  rules,  templates  are  structural  meta-level  knowledge--clearly  meta¬ 
knowledge  of  the  second  type  we  have  described  above.  Thus,  in  MYCiN  at  least,  the 
meta-cognitive  ability  to  say  what  is  known  depends  on  meta-knowledge  of  the 

representation  of  what  is  known.1 

1  It  is  Important  to  emphasize  that  this  representational  meta-knowledge  is  in 
the  domain  of  programming  languages,  though  it  is  is  suggestive  of  the  third  kind  of 
meta-knowledge— knowing  about  the  thinking  process  itself. 
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In  summary,  we  use  the  term  meta-knowledge  in  this  dissertation  to  characterize 
knowledge  of  distinctly  different  content— knowledge  about:  (1)  what  is  known,  (2) 
levels  of  abstraction,  and  (3)  information  processing  capability.  At  the  heart  of  the 
problem  of  distinguishing  among  types  of  meta-knowledge  is  that  all  reasoning 
inevitably  involves  other  facts;  recent  work  in  cognitive  psychology  suggests  that 
knowledge  about  facts  is  as  important  in  problem  solving  as  the  facts  themselves.  If 
this  is  so,  we  may  find  that  we  are  attaching  "meta-"  to  the  name  of  every  cognitive 
operation,  falsely  suggesting  that  there  are  special  kinds  of  Induction,  memory,  etc., 
that  operate  with  "meta-knowledge."  Rather,  the  rise  of  terms  like  meta-induction, 
meta-memory,  and  meta-cognition  reflects  the  importance  of  characterizing  the 
content  of  problem-solving  knowledge,  rather  than  Just  the  mechanism  or  process 
itself. 


In  fact,  this  chapter  has  illustrated  that  knowledge  that  is  useful  for  tutoring 
goes  well  beyond  the  set  of  performance  rules  and  control  scheme  that  make  M YCIN 
an  effective  problem-solver.  In  order  to  teach  effectively,  GUIDON  has  to  reason  about 
MYCIN's  rules.  Besides  simply  being  able  to  access  and  dissect  the  rules,  this 
reasoning  involves  knowing  about  their  complexity,  how  they  are  related,  what 
knowledge  is  necessary  to  understand  them,  and  how  they  should  be  applied.  Thus 
GUIDON  makes  use  especially  of  met a-/evef*  of  knowledge  (type  2  meta-knowledge) 
in  tutorials. 
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Chapter  6 

Dialogue  Management  Heuristics  for  Buie-Based  Tutorials 
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6. 2. 3. 3  Opportunistic  Tutoring 
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>  presenting  an  unknown  rule 
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6.1  Dialogue  Management  Overview 

A  tutorial  program  that  engages  a  student  in  mixed-initiative  discussion  must 
have  the  means  to  manage  its  share  of  the  dialogue.  Managing  a  dialogue  ranges  from 
record-keeping  and  context-focusing  to  heuristics  for  directing  the  dialogue 
economically  according  to  the  needs  of  the  student.  GUIDON'S  expertise  for  carrying 
on  a  case  method  dialogue  is  represented  as  a  network  of  procedures.  The  links 
between  procedures  are  management  heuristics,  in  the  form  of  t-rules,  that  control 
transitions  between  dialogue  situations. 

This  chapter  surveys  GUIDON'S  discourse  procedures  (sequences  of  t-rules).  A 
dialogue  transition  diagram  represents  the  procedural  calling  structure,  revealing  the 
structure  of  GUIDON'S  initiative  during  the  tutorial.  The  penultimate  section  (Section 
6.3)  summarizes  the  principles  that  the  program  follows  when  guiding  the  dialogue. 
We  conclude  that  a  network  of  procedures  Is  a  useful  representation  for  organizing 
heuristics  for  carrying  on  a  structured  dialogue. 

6.1.1  Scope  of  the  Problem 

The  complex  and  time-consuming  task  of  diagnosing  an  infectious  disease 
requires  expertise  for  managing  a  dialogue  that  goes  well  beyond  the  local  response 
strategies  that  have  been  considered  in  other  tutorial  programs.  Sequences  of 
student/tutor  remarks  can  be  structured  into  discourse  situations  or  recurrent 
patterns  in  the  discussion,  making  the  mixed-initiative  nature  of  this  tutorial  more 
complex  than  replying  to  a  single  student  question  and  generating  another  problem. 
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For  example,  with  the  purpose  of  presenting  s  domain  rule  as  a  form  of  assistance 
(the  student  doesn't  know  what  to  consider  next),  the  program  not  only  chooses  the 
rule  to  present,  it  also  selects  a  method  of  presentation  that  accounts  for  the 
complexity  of  the  domain  rule,  what  the  student  knows,  and  what  he  will  have  to  do 
before  he  can  apply  the  rule.  One  action  sequence  is:  (1)  suggest  a  subgoal  to  the 
student;  (2)  discuss  this  subgoal  with  student  (involving  perhaps  a  protracted 
consideration  of  several  other  rules);  (3)  prompt  the  student  for  the  conclusion  that 
can  be  drawn  using  this  new  information;  and  (4)  discuss  related  rules  with  the 
student.  Discourse  knowledge  is  also  important  for  coping  with  the  student's 
initiative.  For  example,  the  program  has  methods  for  deciding  what  to  say  when  the 
student  says  that  he  knows  the  solution  to  a  subtask  (and  doesn't  want  to  discuss  it 
in  detail).  Given  these  dialogue  components  and  conditions  for  following  them,  we  find 
that  a  basic  characteristic  of  a  mixed-initiative  tutorial  is  the  possibility  of  altarnativa 
dialogue*. 

The  WHY  program  [Stevens  !■  Collins,  1977]  has  a  goal  structure  that  ultimately 
provides  organization  to  the  tutor's  choice  of  subject  material,  but  this  is  not  the  kind 
of  knowledge  about  situations  that  we  have  considered  in  GUIDON.  In  GUIDON  we 
have  characterized  alternative  tutorial  actions,  such  as  wrapping  up  discussion  of  a 
rule,  in  terms  of  sequential  considerations  that  comprise  them  (as  illustrated  by  the 
example  above).  On  the  other  hand,  while  we  focus  on  the  problem  of  making 
appropriate,  prolonged  presentations  that  go  beyond  interruption  and  repetitive 
question/answering  (as  In  WHY),  we  use  them  as  components  in  a  larger  scheme. 
Many  of  our  t-rules  bear  resemblance  in  kind  to  tutorial  strategies  that  appear  in 
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WEST  [Burton,  1979]  (e.g.,  a  rule  that  decides  whether  to  interrupt  the  student), 
WHY  (e.g.,  a  rule  that  asks  the  student  for  relevant  factors  to  support  his  wrong 
hypothesis  for  a  goal),  and  WUMPUS  [Goldstein,  1977]  (e.g.,  rules  that  summarize 
evidence  for  a  goal).  The  main  difference  between  GUIDON  and  these  other  programs 
is  that  GUIDON'S  tutoring  rules  are  structured  by  situations  that  occur  in  a 
comprehensive  framework  for  discussing  a  case. 

6.1.2  Management  Constraints 

Dialogue  management  involves  coordinating  tutorial  goals  with  constraints 
imposed  by: 


1 )  the  time  available  for  the  session 

2)  student  initiative  and  conversational  (social)  postulates 
[Gordon,  1971]  [Grice,  1976] 

3)  the  communication  channel 

4)  human  memory  and  learning  capability. 

•% 

In  early  development  of  GUIDON,  the  limitation  of  time  and  verbosity  have  been 
the  chief  forcing  functions:  it  Is  not  possible  to  explicitly  discuss  every  inference  that 
the  expert  program  attempts.  By  the  use  of  heuristics  for  economical  presentation 
and  student  Initiative  options  presented  below,  the  crux  of  dialogue  management  in 
GUIOON,  session  time  for  relatively  simple  cases  has  been  reduced  from  about  five 
hours  to  less  than  one  (for  students  who  know  how  to  use  the  program). 


1  08  DIALOGUE  MANAGEMENT  HEURISTICS 

The  theoretical  constraints  listed  above  cut  across  practical  Issues  of  tutoring. 
These  are  the  issues  that  are  considered  in  this  chapter  and  the  next: 

--  On  what  basis  does  a  tutor  select  alternate  presentation  techniques? 
--  How  can  we  ensure  dialogue  connectedness  and  comprehensibility? 

>-  How  does  a  tutorial  program  maintain  and  share  dialogue  context? 

--  How  can  we  provide  for  and  cope  with  student  Initiative? 

The  first  two  issues,  alternate  presentation  techniques  and  comprehensibility,  are 
considered  in  this  chapter.  Chapter  7  deals  with  sharing  context  and  student 
initiative. 

6.1.3  Knowledge  of  Discourse  Structure 

Guidon's  discourse  procedures  formalize  knowledge  of  discourse  situations  and 
strategies  for  controlling  transitions  between  situations.  This  section  relates  our  work 
to  other  Natural  Language  studies,  describes  the  t-rule  formalism,  and  in  broad  terms 
characterizes  the  situations  and  strategies  we  have  formalized.  Subsequent  sections 
will  detail  the  implementation  of  the  rules  and  survey  their  specific  content. 

6. 1.3.1  Discourse  Patterns 

Recent  Natural  Language  research  has  recognized  that  human  dialogues  exhibit 
regularity  in  the  form  of  patterns  that  encompass  several  turns  of  initiative  and  that 
recur  regularly  [Levin  &  Moore,  1977]  [Bobrow,  et  el.,  1977]  [Bruce,  1976] 
[Deutsch,  1974]  [Winograd,  1977].  To  quote  Bruce  [emphasis  added]: 
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(It  is)  ...  useful  to  have  a  model  of  how  social 
interactions  typically  fit  together,  and  thus  a  model  of 
discourse  structure.  Such  a  model  can  be  viewed  as  a 
heuristic  which  suggests  likely  action  sequences. .  .  .  There  are 
places  In  a  discourse  where  questions  make  sense,  others 
where  explanations  are  expected.  (These  paradigms)  .  .  . 
facilitate  generation  and  subsequent  understanding.  [Bruce, 
1975] 


GUIDON'S  knowledge  for  carrying  on  the  case  method  dialogue  is  structured 
according  to  the  recurring  patterns  that  one  finds  in  rule-based  case  method 
dialogues.  Each  pattern  is  represented  as  a  discourse  procedure  that  is  made  up  of  a 
series  of  steps;  thus,  these  procedures  are  the  action  sequences  mentioned  by 
Bruce.  Their  applicability  Is  determined  by  t-rules. 

Faught  [Faught,  1977]  mentions  two  types  of  dialogue  patterns:  interpretation 
patterns  (to  understand  a  speaker)  and  action  patterns  (to  generate  utterances). 
GUIDON  does  not  currently  use  Interpretation  patterns  because  it  requires  the  student 
to  change  topics  explicitly  using  command-oriented  options.  Discourse  procedures  are 
action  patterns:  they  serve  as  an  ordered  list  of  options— types  of  remarks  for  the 
program  to  consider  making  at  a  particular  time.  For  example,  the  procedure  for 
discussing  a  domain  rule  (d-rule)  includes  a  step  that  Indicates  to  "consider 
mentioning  d-rules  related  to  the  one  just  discussed."  In  this  way,  a  discourse 
procedure  step  specifies  in  a  schematic  form  WHEN  a  type  of  remark  might  be 
appropriate.  WHETHER  to  take  the  option  (e.g.,  is  there  an  "interesting"  d-rule  to 
mention?)  and  WHAT  to  say  exactly  (the  discourse  pattern  for  mentioning  the  d-rule) 
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will  be  dynamically  determined  by  tutoring  rules  whose  preconditions  refer  to 
attributes  of  the  knowledge  being  discussed  and  the  communication  model  (student 
model,  case  lesson  plan,  and  focus  record).  See  Figure  6.1. 


PREMISE 


Domain  Knowledge  Reference 
Communication  Model  Reference 

—  Overlay  Student  Model 
—  Case  lesson  plan 
—  Focus  Record 


ACTION 


DISCOURSE  PROCEDURE 

—  T-rule  Packet 
—  Discourse  Procedure 
—  Primitive  Function 


Figure  6.1.  Form  of  a  Tutorial  Rule 


T-rules  are  actually  grouped  into  packets,  each  specific  to  a  tutorial  goal.1 
There  are  two  types  of  t-rule  packets:  t-rules  for  selecting  a  discourse  procedure 
'  and  t-rules  for  modeling  belief.  A  packet  of  t-rules  for  srfrctm;  a  discourse  procedure  to 
follow  generally  applies  its  t-rules  In  order  and  stops  when  the  first  one  succeeds. 
The  form  of  t-rules  of  this  type  Is  shown  In  Figure  6.1  above.  The  action  part  consists 

of  stylized  code,  just  like  the  steps  of  a  discourse  procedure.2  A  step  may  invoke: 

1)  A  packet  of  t-ruies,  e.g.,  to  select  an  introductory  remark 
for  discussing  a  given  d-rule. 


2)  A  discourse  procedure,  e.g.,  to  sequentially  discuss  each 
precondition  of  a  d-rule. 


1  The  INTERLISP  representation  Is  described  In  Appendix  E. 

2  Discourse  procedure  steps  also  contain  control  information  (e.g.,  for  iteration) 
that  is  not  important  to  this  discussion.  See  Appendix  E. 
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3)  A  primitive  function,  e.g.,  to  accept  a  question  from  the 
student  or  perform  bookkeeping  (see  Figure  E.3  (in  Appendix  E)  for 
a  list  of  these). 

Observe  that  the  action  part  of  a  t-rule  is  a  discourse  procedure  and  discourse 
procedures  invoke  t-rules,  so  the  representation  is  recursive. 

The  second  kind  of  t-ruie  packet  is  for  modeling  belief—  updating  the 
communication  model  and  rank-ordering  d-rules  according  to  an  hypothesis  revision 
strategy  are  two  examples.1  Generally,  a  packet  of  t-rules  of  this  type  is  applied 
exhaustively.  Communication  model  t-rules  are  described  in  Section  8.3.  T-rules 
for  choosing  question  formats  are  described  in  Appendix  B. 

6. 1.3.2  Transitions  for  Alternative  Dialogues 

It  is  useful  to  formalize  more  than  one  way  for  the  tutor  to  achieve  a  goal,  e.g., 
more  than  one  way  to  present  a  d-rule.  Indeed,  this  is  why  there  is  generally  a 
packet  of  t-rules  for  choosing  a  course  of  action.  In  this  section  we  present  the 
concept  of  a/fernaf/ve  dlaloguaa,  different  ways  something  can  be  said  and  possible 
subdialogues  that  can  occur.  The  concept  is  illustrated  by  a  state  transition  diagram 
that  represents  the  invocation  structure  of  the  discourse  procedures  In  GUIDON.  The 
links  represent  control  expressed  by  tutorial  rules  within  the  procedures.  These  links 
signify  choice  points  that  lead  to  alternative  dialogues,  dictated  by  domain  logic, 
economy,  or  tutorial  considerations.  Thus,  these  represent  the  management  decisions 
In  which  the  tutor  takes  the  initiative  to  control  dialogue  situations. 

1  GUIDON  uses  certainty  factors  lor  representing  the  program's  belief  in  something. 
Their  value  ranges  between  -1  and  1,  with  negative  values  signifying  disbelief.  See 
Shortliffe  [Shortliffe  4  Buchanen,  1976]  for  discussion  of  their  implementation  and. 
significance. 
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Figure  6.2  has  been  extracted  from  a  tutorial  session  (using  the  same  case  that 
appears  in  Chapter  3,  and  analogous  to  page  69  of  that  interaction.  Translations  of 
MYCIN  parameters  have  been  simplified.)  The  discussion  has  reached  the  point  of 
considering  clinical  information  (non-iaboratory  data)  to  determine  the  organisms  that 
might  be  causing  the  infection.  The  student  asked  whether  the  patient  has  a  rash, 
and  the  tutor  gave  the  reported  case  data  (lines  6-7).  At  this  point  several  remarks 
were  made  by  the  tutor:  1)  lines  9-16,  the  datum  requested  by  the  student  was 
related  to  the  current  topic  by  reciting  the  path  that  connects  them  in  this  case:  rash 
->  Herpes  Zoster  virus  ->  infection  type  ->  organisms;  2)  lines  20-21,  the  tutor 
focused  on  the  topic  "Herpes  Zoster  virus,”  and  stated  a  final  conclusion;  3)  lines  23- 
24,  the  tutor  explicitly  returned  the  student's  attention  back  to  the  original  topic  of 
determining  the  organisms  causing  the  infection. 
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You  should  now  proceed  to  ask  questions  that  will  enable  you  to 
make  an  hypothesis  about  the  organisms  that  might  be  causing  the 
Infection. 

5  **  RASHES 

Pt538  does  not  have  a  rash. 

Your  question  is  Indirectly  useful  for  determining  the  organisms 
18  that  might  be  causing  the  Infection. 

Whether  J. Smith  has  a  rash  can  be  used  to  determine  whether 
J. Smith  has  symptoms  of  Herpes  Zoster  virus... 

15  which  can  be  used  to  determine  the  type  of  the  infection... 

and  this  will  enable  us  to  determine  the  organisms  that  might 
be  causing  the  Infection. 

28  The  fact  that  J. Smith  does  not  have  a  rash  Is  evidence  that  J. Smith 
does  not  have  symptoms  of  Herpes  Zoster  virus  [RULE369]. 

Back  to  our  discussion  of  the  organisms  that  might  be  causing  the 
Infection. . . . 


Figure  6.2.  Subdialogue  Involving  a  Deeper  Subgoal 

It  now  may  be  helpful  to  look  at  the  dialogue  transition  diagram  (Figure  6.3). 
Each  node  in  this  diagram  stands  for  a  discourse  procedure,  or  sequence  of  t-rules. 
For  example,  when  lines  1  -6  were  printed  In  Figure  6.2,  the  procedure  GOAL  was  being 
followed,  the  basic  procedure  for  discussing  any  MYCIN  goal  which  appears  in  a 
domain  rule.  To  answer  the  question  about  rashes,  the  DATA  procedure  was  applied. 
Whether  or  not  the  patient  has  a  rash  is  a  "deeper  subgoal"  in  that  it  does  not  appear 
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in  any  rule  that  can  determine  the  current  goal,  but  appears  deeper  in  the  AND/OR 
tree  of  goals  and  rules.  The  DATA  procedure  printed  line  7  and  set  up  the  second- 
deepest  subgoal  as  the  new  current  topic  (whether  the  patient  has  symptoms  of 
Herpes  Zoster  virus).  An  arrow  that  loops  back  to  a  node  signifies  that  the  procedure 
that  labels  it  is  called  one  or  more  times,  as  one  might  expect.  For  example,  we  see 
that  one  or  more  "related  rules"  are  presented  at  the  end  of  discussion  of  a  given 
rule.  The  italicized  labels  stand  for  the  basis  of  the  transitions— economy,  domain 
logic,  and  tutoring  goals—these  distinctions  are  described  below. 
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Figure  6.3.  Dialogue  Transition  Diagram 
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Returning  to  our  example,  the  tutor  observed  that  the  new  topic  (Herpes  Zoster- 
-see  lines  12-13  in  Figure  6.2)  was  completed  because  all  of  the  case  data  that  the 
expert  needed  to  make  a  final  conclusion  had  already  been  given  to  the  student. 
Here  a  choice  had  to  be  made:  Should  the  tutor  present  the  final  conclusion  (as  it  did 
in  lines  20-22)?  Or  should  a  summary  of  evidence  be  offered?  Or  should  the  tutor 
ask  the  student  to  make  an  hypotnesis  (as  to  whether  or  not  the  patient  has 
symptoms  of  Herpes  Zoster)?  Figure  6.4  and  Figure  6.6  show  the  procedure 
COMPLETEDGOAL.  We  see  that  T-RULE6.02  was  applicable:  a  single  domain  rule  was 
used  by  the  expert  program  to  make  a  conclusion  and  the  student  model  indicated 
that  the  student  knew  this  rule,  so  it  was  simply  stated.  The  fact  that  the  dialogue 
could  have  taken  a  different  form  at  this  point,  at  the  tutor's  initiative,  illustrates  the 
possibility  of  alternative  dialogues.  There  are  many  more  alternative  dialogues  than 
those  illustrated  by  the  different  paths  in  Figure  6.3.  For  example,  the  procedure  for 
discussing  a  rule  (shown  here  as  RULE)  incorporates  18  different  methods,  e.g., 
clause-by-clause  discussion,  supplying  case  data  and  then  asking  for  an  hypothesis, 
and  discussing  a  failed  subgoal  and  then  mentioning  the  conclusion  that  can't  be 
made.  These  are  described  In  Section  6.2  below. 

COMPLETEDGOAL . PROC005 

Purpose:  Discuss  final  conclusion  for  the  goal  currently  being  discussed 
by  giving  the  final  answer  or  discussing  an  hypothesis 

Stepl:  <Decide  whether  to  finish  with  a  summary) 

StepZ:  <D1scuss  the  final  hypothesis  for  the  goal) 

Step3:  <Wrap  up  discussion  or  record  that  it  is  complete) 


Figure  6.4.  Discourse  Procedure  for  Discussing  a  Completed  Topic 
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T-rule  for  Deciding  Whether  to  Finish  with  e  Summary » 
T-RULE5 .01  { Summarize  if  >/  rule  and  at  least  one  has  not  been  discussed } 


If:  1)  There  are  rules  having  a  bearing  on  this  goal  that  have  succeeded  and 
have  not  been  discussed,  and 

2)  The  number  of  rules  having  a  bearing  on  this  goal  that  have  succeeded 
is  1 

Then:  It  is  true  that  no  summary  of  evidence  should  be  offered  for  this  goal 


T-rules  for  Deciding  How  to  Discuss  the  Final  Hypothesist 
Apply  the  first  rule  that  is  appropriate: 

T-RULE5 .02  [Directly  state  single,  known  rule) 


If:  1)  There  are  rules  having  a  bearing  on  this  goal  that  have  succeeded  and 
have  not  been  discussed,  and 

2)  The  number  of  rules  having  a  bearing  on  this  goal  that  have  succeeded 
Is  1,  and 

3)  There  Is  strong  evidence  that  the  student  has  applied  this  rule 
Then:  Simply  state  the  rule  and  Its  conclusion 

T-RULE5.03  [Request  hypothesis  when  rules  may  be  unfenon/n} 


If:  You  have  examined  the  rules  having  a  bearing  on  this  goal  that  have 
succeeded  and  have  not  been  discussed,  and  have  found  a  rule  under 
consideration  for  which  there  Is  not  strong  evidence  that  the  student 
has  applied  the  rule  under  consideration 
Then:  Substep  1.  If:  1)  An  Introductory  remark  Is  to  be  made  before 

requesting  an  hypothesis  from  the  student,  and 
2)  The  student  has  not  requested  help  for  forming  an 
hypothesis 

Then:  Say:  hypothesis-ready 

Substep  11.  Discuss  the  student's  hypothesis  for  the  goal  currently 
being  discussed  [Proc014] 


T-ruie  for  Ending  the  Discussion  of  the  Goah 
T-RULE5 .04  [Now  summarize  or  exit } 


If:  1)  You  have  decided  that  no  summary  of  evidence  should  be  offered  for 
this  goal,  or 

2)  There  are  no  rules  having  a  bearing  on  this  goal  that  have  succeeded 
Then:  Hark  the  goal  currently  being  discussed  as  discussed 
Else:  Discuss  a  summary  of  evidence  for  the  goal  currently  being 
discussed  [Proc886] 

Figure  6.6.  T-rules  for  Carrying  Out  the  Step*  Shown  In  Figure  6.4 
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The  COMPLETEDGOAL  procedure  Illustrates  two  important  constraints  for 
choosing  among  alternative  dialogues  on  the  basis  of  the  principle  of  economy. 
Specifically,  surface  complexity  features  of  the  knowledge  being  discussed  (e.g.,  the 
number  of  rules  involved,  whether  a  rule  is  definitional,  that  all  relevant  rules  fail,  and 
how  many  subgoals  remain  to  be  completed  before  an  inference  can  be  drawn)  and 
the  student  model  combine  In  these  tutorial  rules  to  select  a  presentation  method  that 
affects  how  extensive  the  discussion  will  be. 

When  a  conservatively  programmed  tutor  is  not  sensitive  to  the  complexity  of  a 
situation  or  the  student's  knowledge,  tedious,  overly  detailed  discussion  results;  the 
program  belabors  topics  that  can  be  dealt  with  swiftly  by  a  single  remark  or  simply 
skipped  over.  For  example,  a  striking  improvement  to  GUIDON  tutorials  was  based  on 
the  recognition  that  many  MYCIN  topics  are  "definitional,"  so  the  usual  process  of 
discussing  all  of  the  evidence  explicitly  and  asking  the  student  to  make  an  hypothesis 
can  be  bypassed  when  the  model  Indicates  that  the  student  knows  the  relevant  rule. 

Besides  economical  considerations,  the  dialogue  transition  diagram  illustrates 
two  other  kinds  of  transitions:  loglcml  and  tutorial.  The  links  leading  from  the  DATA 
procedure  are  distinctions  based  on  domain  logic,  as  are  the  three  straight  links 
leading  from  GOAL.  These  links  are  based  on  domain  facts  and  relations  (e.g.,  that  a 
topic  is  the  name  of  a  block  of  case  data)  and  decisions  made  by  the  expert  program 
(e.g.,  why  a  question  need  not  be  asked).  In  following  these  transitions,  the  tutor  is 
reasoning  about  the  subject  material. 


Tutorial  transitions  are  based  on  the  tutor's  goals  for  teaching  particular  material 
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to  the  student  and/or  refining  its  model  of  his  knowledge.  The  examples  in  Figure  6.3 
are:  1 )  quizzing  the  student  about  rules  that  recently  succeeded  or  failed  because  of 
case  data  just  given  to  the  student  ("recent-success"  and  "-failure"  In  Figure  6.3); 
2)  providing  initial  orientation  to  get  the  student  started  on  a  topic  that  is  new  to  him 
("orient");  and  3)  quizzing  about  rules  related  to  the  one  just  discussed  ("related- 
rules").  We  call  these  instances  of  opportunistic  tutoring  because  they  signify 
initiative  taken  by  the  tutor  at  appropriate  times  with  the  purpose  of  fitting  as  much 
information  into  the  session  as  is  practical.  These  quizzes  constitute  subdialogues 
that  can  be  quite  similar  in  motivation  and  format  to  SCHOLAR  [Collins,  1976],  WHY 
[Stevens  &  Collins,  1977],  and  ABLOCKS  [Brown,  et  al„  1976]  tutorials. 

The  appropriateness  of  the  interruption  is  determined  by  its  consideration  within 
the  sequence  of  a  discourse  procedure,  a  conventional  dialogue  pattern,  and  by 
heuristics  that  determine  whether  a  remark  should  be  made.  The  basic  idea  is  that 
the  prggram  has  certain  topics  that  it  wants  to  bring  up  (e.g.,  a  couple  of  rules 
relevant  to  the  current  case  that  the  student  may  not  know),  so  it  checks  to  see  if 
the  kind  of  remark  that  is  appropriate  at  a  given  time  provides  the  opportunity  to  raise 
a  particular  topic.  Thus,  after  the  tutor  returned  attention  to  the  previous  topic  in 
Figure  6.2  (lines  23-24),  it  checked  to  see  if  there  were  any  rules  that  it  wanted  to 
mention  that  had  just  then  been  applied  by  the  expert  program  (there  were  none). 
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6.2  Rhetorical  Structure  of  the  Dialogue 

This  section  surveys  the  dialogue  situations  of  a  GUIDON  tutorial,  organizing  them 
according  to  the  three  types  of  transitions  that  bring  them  about:  economic*!, 
logical,  and  tutorial  /modeling.  In  this  organization,  we  will  be  emphasizing  the 
underlying  thrust  of  each  shift  in  the  dialogue,  though  particular  t-rules  will  reflect  a 
mixture  of  all  considerations.  For  example,  we  consider  t-rules  for  discussing  a 
domain  rule  as  examples  of  the  tconomic  thrust  of  the  dialogue,  though  naturally  the 
action  of  these  t-rules  will  reflect  the  logic  of  the  domain.  Similarly,  t-rules  that 
present  case  data  should  adhere  to  principles  of  economy,  though  their  thrust  will  be 
to  cope  with  the  domain  logic:  is  a  student’s  question  redundant? 

For  each  dialogue  situation,  we  characterize  (1)  the  tutor's  goal;  (2)  actions 
that  achieve  that  goal;  and  (3)  conditions  under  which  alternative  actions  are  chosen. 
Where  appropriate,  we  will  consider  general  strategies  that  are  incorporated  in  the 
design  of  a  group  of  rules,  for  example  the  general  approach  for  laying  down  a 
foundation  and  for  filling  in  details.  When  there  are  just  one  or  two  rules  that  are 
significant  heuristics  for  controlling  the  dialogue,  they  will  be  described  In  detail. 
Otherwise  a  reference  will  be  given  to  a  procedure  listed  In  Appendix  F. 

The  reader  should  keep  in  mind  that  the  t-rules  shown  here  are  intended  to 
illustrate  plausible.  Interesting  dialogues,  not  to  advocate  particular  teaching  methods. 
While  there  are  in  fact  some  general  principles  that  we  tend  to  favor,  the  point  of  this 
exposition,  indeed  the  point  of  the  initial  implementation  of  GUIDON,  is  to  exemplify 
what  strategies  are  possible,  rather  than  to  prove  claims  about  what  is  desirable.  The 
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tutorial  principles  used  by  GUIDON  are  summarized  in  Section  6.3  at  the  end  of  this 
chapter. 


6.2.1  Transitions  li  Economy 

Recall  that  dialogue  transitions  based  on  economy  use  information  about  the 
surface  complexity  of  the  knowledge  being  discussed  and  the  student  model  to 
appropriately  direct  the  dialogue.  These  management  decisions  reflect  the 
conversational  postulate  that  one  should  be  perspicuous  [Grice,  1976]  [Bruner, 
1966]  or  "to  the  point."  It  all  boils  down  to  giving  GUIDON  more  than  one  way  to  do 
something,  so  that  the  program  can  tailor  what  it  says  to  the  specific  situation  at 
hand.  For  example,  the  program  would  have  run  perfectly  well  if  it  were  only  able  to 
discuss  a  domain  rule  by  taking  each  clause  in  sequence.  Yet  when  all  but  one 
subgoal  has  already  been  discussed  (in  another  context),  why  not  simply  discuss  that 
one  subgoal  and  mention  the  other  information  as  an  aside?  This  is  the  kind  of 
alternative  dialogue  that  t-rulea  based  on  economy  bring  about. 

6.2. 1.1  Economically  Discussing  a  D-rule 

In  entering  this  situation,  the  tutor  has  already  decided  to  discuss  the 
application  of  a  particular  d-rule  to  the  case  at  hand.  GUIDON  has  18  methods  for 
presenting  a  d-rule;  they  are  naturally  categorized  by  whether  they  present  d-rules 
that  fail  or  d-rules  that  succeed.  Thus,  the  tutor's  goal  in  discussing  a  d-rule  is  to 
present  an  inference  that  may  be  made  about  the  current  case,  or  one  that  might 
have  been  made  if  conditions  had  been  different.  Discussing  a  successful  rule 
cleuse-by-clause  is  sufficiently  complex  to  warrant  separate  mention. 
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Discussing  a  Rule  Clause-by-Clause 

The  protocol  (page  82)  illustrates  how  a  rule  that  succeeds  is  discussed  one 
clause  at  a  time  (see  RULE.PR0C001 ).  The  main  steps  are  to  announce  the  rule, 
present  an  initial  quiz  about  the  rule,  consider  each  subgoal  in  order,  and  then  discuss 
the  rule's  conclusion.  This  last  step  is  a  discourse  procedure  itself 
(ENDOFRULE.PROCOI  7)  that  involves:  (1)  presenting  the  conclusion;  (2)  commenting 
upon  possible  circular  reasoning,  previous  evidence,  and  the  significance  of  this 
inference;  (3)  describing  the  completion  status  of  this  goal;  (4)  presenting  related 
rules;  (5)  announcing  the  completion  of  discussion  of  this  rule.  This  detailed  method 
for  discussing  a  rule  is  generally  used  in  GUIDON  when  the  rule  is  syntactically 
complex  (e.g.,  it  contains  an  iterative  precondition)  or  the  tutor  wants  to  draw  out 
discussion  of  the  rule  (e.g.,  to  present  an  important  rule  that  the  student  does  not 
know). 

Discussing  Rules  that  Succeed:  Specific  Situations 

This  rule  packet,  SUGGESTRULE.RULEPK031,  uses  the  following  general  methods 
for  presenting  a  rule  that  successfully  applies  to  the  case: 

1 )  Posa  tha  rula  a s  •  quastlon,  possibly  giving  new  case  data. 

2)  Discuss  a  subgoal,  and  then  discuss  the  rule  clause-by- 
cJause  or  wrap  up  discussion  of  the  rule  immediately. 

3)  Relate  the  rule  to  the  context  of  the  discussion,  then 
discuss  tha  rula  clausa-by-clausa. 

4)  Posa  a  hint  about  the  rule's  effect  on  the  hypothesis 
discussed  so  far. 
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The  conditions  for  selecting  among  these  alternative  dialogues  are  summarized 
below: 


1)  The  status  of  the  rule's  application.-  Are  there  enough 
case  data  to  evaluate  It?  What  (how  many)  case  data  are  missing? 
What  (how  many)  subgoals  (factors  that  Involve  rule  application) 
remain  to  be  considered? 


2)  The  recent  context  of  the  Interaction  (focus  record): 
mention  of  a  deeper  subgoal,  a  hint  about  this  rule,  or  mention  of  a 
factor  that  appears  in  this  rule. 

3)  Properties  of  the  rule:  whether  a  precondition  is  an 
iteration,  whether  the  current  topic  is  a  true/false  parameter. 

4)  The  student  model:  does  the  student  know  this  rule? 

Discussing  Rules  that  Fail 

T-rules  for  presenting  an  inference  that  cannot  be  made  are  quite  similar  to  t- 
rules  for  presenting  successful  rules.  However,  there  are  several  special 
preconditions  that  are  of  interest: 


1)  A  d-rule  is  said  to  fail  inappropriately  (for  the  purpose  of 
forming  it  into  a  question)  if  there  are  factors  in  its  precondition 
that  didn't  need  to  be  pursued  by  MYCIN.  This  occurs  when  the 
interpreter  determined  that  the  rule  failed  before  evaluating  all 
preconditions,  and  skipped  preconditions  that  weren't  evaluated  in 
other  rules  either.  For  example,  the  rule  may  have  failed  because 
the  infection  is  cystitis,  so  MYCIN  never  needed  to  determine 
whether  the  patient  had  certain  symptoms  of  that  infection. 
Because  facts  are  "missing,"  it  is  not  possible  to  fit  the  rule  into 
one  of  the  question  formats  available  to  the  program;  the  current  t- 
rule  that  handles  this  case  simply  says  that  certain  information  is 
relevant  without  going  into  details  (t-rule23.03). 


2)  It  is  especially  interesting  when  a  d-rule  fails  because  a 
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single  patient  datum  was  unknown  at  the  time  of  the  consultation. 
GUIDON  chooses  a  question  format  that  singles  out  this  information 
(t-rule23.04). 


3)  If  a  d-rule  has  an  undetermined  subgoal  that  is  also  useful 
for  d-rules  that  succeed,  it  is  worthwhile  to  discuss  this  subgoal  (t- 
rule23.06). 

Recap;  How  the  Dialogue  Situation  Determines  What  is  Economical 

The  rule  packets  for  discussing  inferences  clearly  illustrate  how  economical 
management  of  the  dialogue  stems  from  a  collection  of  procedures  for  dealing  with 
specific  situations.  From  this  we  conclude  that  guiding  the  dialogue  in  an  interesting, 
or  efficient  way  involves  (at  least)  an  analysis  of  the  superficial  complexity  of  the 
knowledge  being  discussed.  This  is  a  domain-independent  level  of  abstraction  that 
deals  with  conditions  we  have  listed  above,  e.g.,  whether  a  subgoal  appears  in  other 
rules,  whether  more  d-rules  need  to  be  considered  before  this  inference  can  be 
attempted,  how  many  subgoals  remain  to  be  considered,  and  so  on. 

It  appears  that,  for  a  rule-based  tutorial  of  the  type  we  have  formalized,  the 
flow  and  structure  of  the  dialogue  Is  intimately  tied  to  the  abstract  properties  of  the 
problem's  decomposition  into  successful,  failed,  partially  complete,  and  unknown 
subgoals  and  Inferences.  That  this  would  be  so  is  not  surprising,  given  that  the 
purpose  of  the  dialogue  is  to  "fill  In"  the  abstract  solution  tree  for  the  problem.*  it  is 
not  surprising  that  what  will  be  interesting,  what  nuds  to  b<  said,  is  directly  determined 
by  characterizing  what  remains  to  be  done  before  the  solution  is  complete.  Thus 
GUIDON'S  t-rules  detect  the  holes  and  fill  them  in.  Each  t-rule  makes  a  small 
contribution  by  supplying  a  few  more  pieces  of  case  data,  wrapping  up  consideration 
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of  a  subgoal,  and/or  presenting  an  inference:  in  all  cases  moving  up  the  tree  In 
relatively  simple,  easy  to  follow  steps. 

6.2.1. 2  Economically  Discussing  a  Topic 

Just  as  the  long  way  to  discuss  a  d-rule  is  to  discuss  each  of  its  clauses,  the 
long  way  to  discuss  a  topic  is  to  discuss  each  of  the  rules  that  conclude  about  it.  It 
is  here  that  the  time  element  of  a  dialogue  is  critical:  A  single  topic  may  involve  1 00 
rules  and  It  might  require  an  hour  to  get  through  them  all  if  they  are  all  discussed. 
Moreover,  no  student  is  likely  to  sit  still  through  a  tedious  consideration  of  the  dozens 

t 

of  inferences  that  had  no  bearing  on  the  current  case.  Many  rules  may  deal  with 
special,  rare  cases  that  are  of  academic  interest;  others  may  have  been  considered 
in  previous  tutorials  and  not  require  review. 

Discussing  Only  Sufficient  Evidence 

For  these  reasons,  GUIDON  does  not  simply  iterate  over  the  list  of  rules  for  a 
topic.  Instead,  discussion  of  a  goal  focuses  on  the  relevant  evidence  to  be  collected. 
Options  discussed  in  Chapter  7  are  designed  to  make  it  easy  for  the  student  to 
display  lists  of  evidence  to  consider  and  to  explore  its  use  in  the  solution.  GUIDON 
will  be  able  to  examine  the  student's  understanding  when  he  states  hypotheses.  The 
student  model  is  updated  then,  and  this  Is  when  the  program  decides  which  of  the 
Inferences  require  detailed  discussion.  Note  that  GUIDON  does  not  attempt  to  follow 
the  student's  thinking  by  constructing  a  model  to  account  for  the  case  data  he 
requests  and  the  order  In  which  he  requests  it.  See  Section  8.1  for  further 


discussion. 
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While  there  are  generally  fewer  pieces  of  case  data  to  consider  than  rules  that 
use  them,  it  is  still  impractical  to  require  the  student  to  explicitly  request  each  fact, 
or  to  wait  for  him  to  use  one  of  the  options  that  print  blocks  of  information.  For  this 
reason,  GUIDON  uses  a  crude  heuristic  for  detecting  that  sufficient  evidence  has  been 
collected  to  warrant  an  end  to  the  discussion.  The  current  version  of  this  t-rule  (t- 
rule  1.11)  compares  the  expert  program's  current  hypothesis  to  the  final  hypothesis 
that  it  will  reach  once  all  evidence  is  considered.  If  all  of  the  significant  positive  and 
negative  conclusions  are  in  agreement,  then  the  student  is  told  that  discussion  is 
substantially  complete  and  asked  whether  he'd  like  to  move  on  to  another  topic.  If  he 
agrees,  then  the  procedure  for  completing  discussion  of  a  goal  is  invoked.  A  better  t- 
rule  might  take  into  account  the  tutor’s  goals  for  covering  particular  domain  facts  and 
relations  during  this  tutorial  session. 

Discussing  Only  a  Single  Rule 

If  the  rules  for  determining  a  goal  are  mutually  exclusive,  as  they  often  are, 
there  is  no  need  to  consider  each  rule  in  every  case  dialogue.  Other  cases  will 
illustrate  the  other  possible  outcomes.  Instead  of  presenting  all  of  the  other  rules,  we 
might  choose  to  review  one  or  two  "related"  rules  to  prepare  the  student  for  more 
difficult  cases. 

Rather  than  annotate  (or  try  to  detect)  the  condition  of  mutually  exclusive  rules, 
we  have  chosen  to  implement  a  simpler  heuristic  (t-rule  1.02)  that  has  more  general 
applicability.  We  say  that  a  topic  is  known  by  definition  if  it  is  determined  (in  the  case 
under  discussion)  by  a  single  inference  that  makes  a  definite  conclusion  (CF  *  1 .0).  If 
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there  are  other  rules  for  this  topic,  each  either  failed  or  was  not  tried.  In  either 
event,  there  Is  reason  to  deal  with  these  rules  as  we  would  deal  with  a  mutually 
exclusive  set.  A  second  reason  for  considering  the  single  successful  rule  alone  Is 
that  definitional  rules  tend  to  be  easy  to  remember,  so  chances  are  good  that  the 
student  will  be  willing  to  dismiss  this  topic  quickly  once  he  Is  shown  or  remmrtnu  <•< 
relevant  rule.  (See  Section  6.1.3  for  related  discussion.) 

The  procedure  for  discussing  a  definitional  rule  (DEFNRULE.RULEPK042)  uses  the 
student  model  and  state  of  completion  of  the  inference  to  decide  whether  to  just 
state  the  conclusion,  pose  the  d-rule  as  a  question,  or  discuss  It  clause-by-clause. 
The  economical  presentation  methods  used  here  incorporate  the  t-rules  for  presenting 
conclusions  during  discussion  of  a  student  hypothesis  (PRESENTVALS.RULEPK026). 

6.2. 1.3  Economically  Completing  Discussion  of  a  Topic 

The  t-rules  shown  in  Figure  6.5  presented  earlier  are  designed  to  complete 
discussion  of  a  topic.  The  tutor's  goal  is  to  make  sure  that  the  student  knows  the 
conclusion  that  can  be  drawn  from  the  relevant  case  data  and  that  he  realizes  that 
the  discussion  is  complete.  The  t-rules  used  here  reflect  economical  guidance  of  the 
dialogue  by  their  choices  concerning  how  the  conclusion  is  discussed  and  whether  a 
summary  of  evidence  is  offered.  There  are  of  course  subtle  Issues  Involved  in 
determining  whether  the  student  really  understands  MYCIN's  conclusion  and  really 
agrees  that  no  more  remains  to  be  said  about  this  topic.  Modeling  the  student's 
understanding  is  accomplished  by  subprocedures  described  elsewhere  (Chapter  8). 
GUIDON  has  no  capability  to  argue  about  factors  that  the  student  wants  to  consiri*>< 
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that  are  not  reflected  in  MYCIN's  rules,  so  after  the  conclusion  Is  discussed  and 
summary  considered,  GUIDON  simply  says  the  topic  Is  closed  and  moves  on  (see,  for 
example,  page  64). 
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6,2.2  Transitions  II:  Domain  Logic 

Dialogue  transitions  based  on  domain  logic  are  forced  upon  the  tutor  by  a 
combination  of  the  facts  and  relations  of  the  domain  and  the  program's  adherence  to  a 
goal-directed  dialogue.  There  are  three  situations  that  the  tutor  must  detect  and 
cope  with:  1)  requests  for  data  that,  from  MYCIN’s  viewpoint,  should  not  be 
requested;  2)  requests  for  data  that  implicitly  change  the  topic;  3)  completion  of 
discussion  (when  all  data  have  been  requested).  We  will  consider  improper  requests 
for  data  and  topic  termination  here  because  they  are  rooted  in  domain  facts  and 
relations;  we  will  consider  implicit  changes  of  the  topic  in  Section  7. 1.1. 2 
because  it  is  basically  a  problem  of  sharing  and  maintaining  dialogue  context. 

6.2.2. 1  Coping  with  'Improper'  Case  Data  Requests 

Naturally,  if  the  student  requests  information  we  should  assume  that  he  has  a 
good  reason  for  asking.  When  we  say  that  a  request  is  improper,  we  mean  that  MYCIN 
does  not  need  the  requested  information  at  this  time,  and  may  in  fact  never  need  it. 
GUIDON  makes  no  attempt  to  account  for  an  improper  request;  in  full  generality,  this 
might  require  forming  new  (plausible)  domain  rules  that  model  the  student's 
understanding  (see  Section  8.5).  Instead,  GUIDON'S  goal  is  to  tell  the  student  why 
a  data  request  is  improper.  The  possible  reasons  are  fisted  below. 

1 )  The  request  Is  premature-.  MYCIN  will  eventually  ask,  but 
first  wants  to  try  to  deduce  the  information  from  rules.  That  is, 
there  are  other  relevant  case  data  that  should  be  considered  (of 
more  general  interest)  that  may  indicate  that  this  specific  question 
is  unnecessary.  If  there  is  a  parameter  that  screens  this  question, 
and  it  needs  to  be  asked,  then  GUIDON  presents  the  screening 
information  (t-rule  32.04).  The  originally  requested  data  wili  be 
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given  by  GIVENPARMDATA.PROC012  at  this  time,  if  it  is  no  longer 
premature.  (This  is  the  case  when  the  screening  parameter 
appears  in  the  only  rule  that  needed  to  be  considered  before  a 
question  could  be  asked.) 


2)  The  request  is  Inappropriate  tor  a  consultant:  MYCIN 
never  asks  for  this  information  (t-rule32.02). 


3)  The  request  Is  unnecessary  for  this  case:  MYCIN  decided 
not  to  ask  or  determined  the  value  by  applying  a  definitional  rule 
(EXPERTDIDNTASK.RULEPK033). 


4)  The  request  is  relevant,  but  not  useful:  MYCIN  asked  for 
the  value,  but  given  the  order  In  which  the  student  has  acquired 
information,  he  does  not  need  to  know  the  value.  That  is,  all  rules 
that  use  this  Information  can  be  evaluated  on  the  basis  of  what  is 
already  known  (t-rule  12.07). 


6. 2.2. 2  Detecting  Completion  of  a  Topic 


Discussion  of  a  topic  ends  when  there  are  no  more  relevant  case  data  for  the 
student  to  collect.  Three  situations  occur: 


1)  All  of  the  rules  for  determining  the  goal  have  been 
evaluated  by  MYCIN  and  a  final  conclusion  Is  In  hand  (t-rule  1 .09). 


2)  There  are  no  rules  to  apply  for  this  topic  in  this  case.  For 
example,  If  the  case  is  about  an  infectious  disease  patient,  there 
may  not  be  cultures  with  organisms  growing  on  them,  thus  there  are 
no  unknown  organisms  to  identify.  Any  rule  that  mentions 
parameters  of  unknown  organisms  will  not  even  be  tried  by  MYCIN 
because  there  Is  no  "context"  in  which  to  evaluate  the  rule  (see  T- 
rulel.01 ). 


3)  Rules  were  tried,  but  all  failed.  While  we  could  let  the 
student  go  ahead  and  collect  relevant  data,  in  the  interest  of 
economical  discussion  GUIDON  simply  gives  the  data  to  the  student 
all  at  once  (ALLRULE SFAILPARM.PR0C036).  The  Implicit  strategy  is 
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that  it  will  make  more  sense  to  discuss  this  topic  in  detail  when 
inferences  can  be  drawn.  (But  note  that  this  t-rule  is  flawed 
because  it  may  be  advantageous  in  certain  situations  to  let  the 
student  discover  for  himself  that  no  inference  can  be  made.  T- 
rule  1 .03  could  be  modified  to  take  this  into  account.) 
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6.2.3  Transitions  Ml:  Tutoring  Goals 

The  point  of  t-rules  based  on  tutoring  goals  is  to  actively  seize  the  initiative  to 
present  specific  rules  and  relations  among  them.  These  t-rules  will  frequently  call  on 
t-rules  based  on  economy  to  keep  the  dialogue  moving  and  present  Information  in  easy 
to  digest  chunks.  They  include  basic  teaching  strategies  that  structure  the 
presentation  of  new  material,  as  well  as  problem-solving  strategies  for  solving  a  case. 
GUIDON'S  t-rules  of  this  type  fall  Into  four  categories:  general  orientation,  providing 
help  to  work  out  the  problem,  opportunistically  examining  understanding,  and 
responding  to  a  student's  hypothesis. 

6.2.3. 1  Tutoring:  Orientation 

By  orientation  we  mean  conveying  the  structure  of  the  knowledge  base  to  the 
student  by  establishing  a  foundation  for  acquiring  specific  rules.  To  a  certain  extent, 
we  have  based  the  t-rules  and  student  options  on  a  general  approach  that  we  feel  is 
useful  when  learning  about  a  new  MYCIN-formalized  domain.  This  approach  is  to  be 
contrasted  with  trying  to  learn  the  rules  by  reading  them  in  an  unorganized  listing.  We 
offer  a  refinement  strategy  that  is  based  upon  the  underlying  inference  structure  of 
the  domain,  as  follows. 

1 )  What  is  the  goal  rule? 

2)  What  is  the  main  subgoal  structure  (sketch  of  a  typical 
solution  tree)? 


3)  What  do  premises  of  rules  look  like?  Are  there  patterns? 
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4)  What  are  the  important  input  data?  What  kinds  of 
judgments  does  the  user  have  to  make? 


5)  What  are  some  typical  values  (outcomes)  for  the  major 
subgoals  and  what  are  some  typical  rules  to  conclude  about  these 
values? 

These  organizational  concepts  and  relations  are  designed  to  help  a  student  grasp  the 
contents  of  an  existing  knowledge  base.1  The  power  of  this  approach  lies  in  the 
Implicit  use  of  an  abstract  solution  tree  that  can  serve  to  hold  in  place  the  various 
organizing  relations:  goal  rule  anchored  at  the  top;  intermediate  concepts  scattered, 
but  tied  together,  throughout;  and  typical  inferences  to  give  substance  to  the 
judgmental  concepts  (typical  data  and  typical  outcomes).  The  student  is  freed  from 
having  to  consider  specific  rules;  instead  he  is  told  only  typical,  common,  or  important 
associations. 

The  extent  to  which  GUIOON  follows  this  orientation  strategy  is  described  below. 
Presenting  the  Goal  Rule  and  Sketch  of  the  Tree 

Every  MYCIN-like  system  has  a  rule  that  stands  at  the  head  of  the  solution  tree. 
It  states  the  task  to  be  completed  ("determine  the  therapeutic  regimen  for  the 
patient")  and  the  major  subgoals  ("determine  the  organisms  that  require  therapy," 
"consider  secondary  infections").  To  carry  out  the  orientation  strategy,  GUIDON’S 
procedure  for  discussing  a  goal  treats  the  top  goal  of  the  solution  in  a  special  way. 
The  following  t-rule  states  the  goal  rule  (substep  i.)  and  prints  the  sketch  of  the 
subgoal  tree  (substep  ii.)  (see  Appendix  D). 

1  It  is  Interesting  to  note  that  they  may  be  useful  guidelines  for  helping  a  human 
expert  to  formalize  what  he  knows  so  that  we  can  fit  it  into  the  MYCIN  representation 
scheme. 
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Step4 :  T-RULE1 .04 


If:  1)  The  goal  currently  being  discussed  is  the  goal  of  the 
consultation,  and 

2)  A:  The  student's  estimation  of  his  sophistication  (on  a  scale 
from  0  to  4)  is  less  than  3,  or 

B:  The  goal  of  the  consultation  has  not  been  discussed  before 
Then:  Substep  1.  Say:  desc-goalrule 

Substep  il.  Show  a  sketch  of  the  subgoal  tree  for  the  goal 
currently  being  discussed 
Substep  ill.  Say:  tel 1-optionname 

Substep  1v.  Discuss  the  goal  rule  of  the  task  clause-by-clause 
[ProcB04] 

Substep  v.  Say:  goodbye 

Substep  vl.  Record  that  the  goal  of  the  consultation  has  been 
mentioned  in  the  dialogue 
Substep  vil.  Exit  this  procedure 

Figure  6.6.  T-rule  for  Discussing  the  Top  Goal  (from  GOAL.PROCOOI) 


Substep  i.  invokes  a  procedure  that  states  the  goal  rule  in  a  case-independent 
terms.  One  problem  here  is  that  MYCIN's  goal  rule  has  evolved  to  an  awkward  form 
that  needs  to  be  greatly  simplified  if  it  is  to  be  of  much  value  to  a  new  student.  This 
could  be  done  by  collapsing  the  clauses  into  a  few  new  intermediate  concepts  like 
"collect  basic  patient  data." 


Substep  ii.  invokes  a  procedure  that  recursively  prints  subgoals  of  the  entire 
solution  tree,  showing  only  those  that  are  "important."  The  current  heuristic  is  that  a 
subgoal  is  important  If  the  number  of  rules  In  which  it  appears  (rules  that  use  it  or 
conclude  it)  exceeds  a  prespecified  threshold:  10  in  the  MYCIN  system.  The  resulting 
"sketch  of  the  subgoal  tree"  is  intended  to  bring  out  the  topics  that  will  play  a 
prominent  part  in  the  dialogue:  subgoals  that  are  concluded  by  a  large  number  of  rules 
will  probably  require  more  time  to  discuss,  and  subgoals  that  are  used  by  a  large 
number  of  rules  will  frequently  enter  Into  the  discussion  of  other  topics.  Therefore,  by 
considering  these  subgoais,  the  student  wilt  have  a  good  beginning  for  evaluating  his 
grasp  on  the  problem. 
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Assuming  that  the  student  wants  to  use  MYCIN's  approach  as  a  starting  point  for 
evaluating  his  understanding,  the  sketch  of  the  tree  might  lead  him  to  ask  specific 
questions  that  will  get  him  started  on  the  problem,  e.g-,  "What  does  'type  of  the 
infection'  mean?"  "What  will  the  patient's  history  of  surgery  tell  me  about  organisms 
causing  an  infection?"  To  a  beginning  student  who  has  very  little  knowledge  of  his 
own  to  go  on,  this  sketch  can  serve  as  a  map  of  the  solution,  and  he  can  refer  to  it  as 
a  checklist  for  filling  in  a  comprehensive  understanding  of  the  expert's  handling  of  the 
case.  For  this  reason,  the  FACTORS,  SUBGOALS  and  PENDING  options  are  available  at 
any  time  during  the  dialogue  to  provide  similar  orientation  information  on  a  slightly  more 
detailed  level  (see  Section  7. 1.4.1).  Substep  iii.  of  t-rule  1.04  tells  the 
student  about  these  options. 

The  strategy  described  here  has  one  basic  flaw:  It  will  probably  overwhelm  the 
student  with  too  much  Information  about  MYCIN's  solution,  and  it  will  probably  frustrate 
any  student  who  knows  enough  so  that  he  could  have  gotten  started  on  his  own.  The 
orientation  strategy  may  have  merit,  but  clearly  3  or  4  pages  of  introductory  text  is 
excessive.  One  Improvement  would  be  to  ask  the  student  if  he  wants  to  see  the  goal 
rule  and  subgoal  sketch,  or  perhaps  show  them  the  first  time  the  student  requests 
assistance. 

Providing  Orientation  for  an  Arbitrary  Goal 

The  strategy  for  providing  orientation  for  a  new  topic  is  to  show  the  student 
what  typical  rule  premises  look  like  (by  listing  the  rule  model)  and  to  list  typical  values 
or  hypotheses  that  one  might  consider.  The  rule  model  is  similar  to,  but  not 
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necessarily  equivalent  to,  the  subgoal  sketch  that  appears  below  this  goal.  The 
typical  values  are  determined  by  a  batch-process  procedure  that  examines  the  rule 
set  and  statistically  ranks  the  values  that  actually  appear  (this  is  stored  on  the  file 
with  other  rule  and  parameter  annotations).1  The  t-rule  that  brings  this  all  about  is 
shown  in  Figure  6.7. 


Step? :  T-RULE36.02 


If:  1)  The  number  of  rules  that  conclude  about  the  current  goal  is 
greater  than  5,  and 

2)  The  goal  currently  being  discussed  has  not  been  discussed 
before 

Then:  Substep  1.  Give  a  fixed-text  definition  for  the  goal  currently 
being  discussed 

Substep  ii.  Provide  orientation  for  finding  out  about  the  goal 
currently  being  discussed  by  listing  the  subgoals 
most  commonly  pursued  in  this  context 

Substep  ill.  Say:  orient-by-rule 

Substep  1v.  List  (typical)  values  for  the  goal  currently  being 
discussed 

Substep  v.  Record  that  the  goal  currently  being  discussed  has 
been  mentioned  in  the  dialogue 

Figura  6.7.  Orientation  for  a  Mew  Topic  (from  ORIENTATION.PROC036) 


Substep  iii-  of  this  t-ruie  presents  statistics  about  the  number  of  d-rules  that 
conclude  about  this  goal  (protocol  page  47).  This  remark  is  intended  to  indicate  to 
the  student  how  much  MYCIN  knows  about  this  topic,  and  thus  forewarns  him  about  the 
lengthiness  of  the  ensuing  discussion.  Information  about  the  extent  of  knowledge  to 
be  learned  is  a  form  of  meta-knowledge  like  that  Investigated  by  Flavell  [Flavell, 
1 077].  The  pedagogical  principle  to  be  tested  Is  that  humans  find  meta-descriptions 


1  The  list  of  "legal  values"  for  a  parameter  is  not  suitable  because  it  may 
include  rare  outcomes  that  are  not  relevant  to  MYCIN's  area  of  expertise.  For 
example,  only  half  of  MYCIN's  list  of  organisms  are  mentioned  in  the  meningitis  and 
bacteremia  rules.  This  list  Is  useful  for  modifying  the  rules  in  quizzes;  rare  values  will 
be  obviously  incorrect  in  a  multiple  choice  question.  Indeed,  the  program  appeared  to 
be  making  jokes  before  we  restricted  values  to  those  that  actually  appeared  in  the 
rules. 
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like  this  to  be  useful  orientation  for  learning  new  material  and/or  carrying  on  a 
prolonged  dialogue. 

Input  data  and  rules 

The  student  can  see  what  case  data  are  relevant  to  a  goal  in  general  by  using 
the  FACTORS  options.  The  CONCLUDE  option  will  describe  the  evidence  that  supports 
(or  contradicts)  a  particular  hypothesis.  These  options  are  discussed  further  in 
Section  7.1.4. 

6. 2.3. 2  Tutoring:  Providing  Help 

The  HELP  option  is  interpreted  to  mean  "suggest  something  that  will  help  me 
complete  the  current  subtask."  GUIDON'S  strategies  for  providing  help  are 
represented  by  two  procedures,  one  for  providing  help  specific  to  a  relevant  factor 
(SPECIFICHELP.PROC020)  and  another  for  providing  general  assistance 
(HELPFORGOAL.PROCOI  6). 

Specific  Help 

Help  specific  to  a  particular  relevant  factor  (e.g.,  "help  me  see  how  information 
about  the  patient's  head  wound  is  Incorporated  in  the  solution)  is  provided  by  a 
sequence  of  t-rules  that:  1 )  provide  case  data  if  appropriate;  2)  describe  hypotheses 
that  the  factor  supports  (USE  option);  3)  open  up  discussion  of  the  factor  (if  it  has 
undiscussed  rules);  and/or  4)  discuss  the  final  conclusion  and  summary  of  evidence. 
This  discourse  procedure  illustrates  that  one  aspect  of  tutoring  Involves  methodical. 
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step-wise  consideration  of  the  solution  process.  GUIDON  follows  a  logical  procedure 
for  making  sure  that  the  student  makes  the  right  connections:  1 )  Does  he  know  what 
case  data  have  been  reported?  2)  Does  he  know  why  it  is  important  to  have  this 
information?  3)  Has  he  considered  how  to  work  out  this  subtask?  4)  Does  he  know 
the  answer  and  what  evidence  led  to  it? 

Observe  that  help  specific  to  a  subtask  of  the  current  topic  is  very  much 
restricted  in  focus:  the  tutor  looks  down  a  subtree  and  "fastens"  it,  if  you  will,  to  the 
broader  context  of  the  current  topic.  General  help,  of  the  "what  should  I  do  next" 
variety,  requires  an  entirely  different  strategy. 

General  Help 

GUIDON  has  a  three-pronged  attack  for  helping  the  student  complete  the  current 
goal.  First,  If  the  dialogue  has  not  recently  focused  on  a  factor  or  hypothesis  (focus 
record  is  empty)  and  the  student  has  not  yet  seen  the  factors  for  this  goal,  then  all  of 
the  factors  tor  this  goal  are  Usted  (t-rule  16.0t).  (This  is  the  FACTORS  option;  see 
Section  7.1. 4.1.)  The  idea  is  that  it  Is  better  to  have  the  student  look  over  the 
list  of  questions  and  subgoals  he  might  pursue,  than  for  the  tutor  to  immediately  seize 
control  by  getting  the  student  started  on  a  particular  solution  step.  Thus,  the 
principle  is  to  leave  the  initiative  In  the  hands  of  the  student  whenever  possible. 

The  second  strategy  is  to  look  for  snags  In  the  problem  solution,  and  so  get  the 
student  past  a  difficult  point.  GUIDON  looks  for  one  particular  snag:  the  presence  of 
"second  thought"  (self-referencing)  rules  that  cannot  be  considered  until  the  "normal" 
inferences  are  all  taken  Into  account.  This  t-rule  (t-rule  16.02)  has  a  very  definite, 
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MYCIN-related  function:  GUIDON  may  be  waiting  for  MYCIN  to  give  the  signal  that  the 
topic  is  done,  but  MYCIN  is  sitting  still  with  one  or  more  successful  inferences  (self- 
referencing  rules)  that  will  not  be  allowed  through  until  some  esoteric  evidence  that 
caused  (normal)  rules  to  fail  is  considered.  This  is  a  very  likely  time  for  the  student  to 
be  confused  about  what  to  do  next. 

To  get  past  this  snag,  GUIDON  presents  each  of  the  (failing)  rules  that  MYCIN 
wants  to  have  evaluated.  The  resulting  interlude  is  short  because  there  are  usually 
just  a  few  rules  of  this  type  and  economical  presentation  is  used  (never  clause-by¬ 
clause).  Of  course,  it  would  be  possible  for  GUIDON  to  tell  MYCIN  to  skip  these  rules, 
effectively  pretending  that  part  of  the  solution  tree  doesn't  exist.  However,  we  took 
a  conservative  approach  in  the  initial  implementation  to  see  what  would  happen  if  the 
tutor  and  student  were  forced  to  consider  each  piece  of  case  data  acquired  by  MYCIN 
during  its  consultation.1  As  long  as  there  are  just  a  few  falling  rules  to  present,  the 
"snag  t-rule"  probably  will  not  need  improvement. 

The  third  strategy  for  providing  general  help  Is  to  select  and  present  a  d-rule 
relevant  to  the  current  task.  The  Idea  is  that  the  most  direct  way  to  help  the  student 
complete  the  current  task  is  to  get  him  started  on  a  particular  inference  for  solving  it. 
The  presentation  strategies  were  presented  in  Section  6.2.1. 1;  it  is  the  selection 
strategies  that  are  of  special  interest  to  us  now. 

To  select  a  rule,  GUIDON  first  uses  the  focus  record  to  select  a  relevant  subset 
of  the  d-rules  for  determining  the  current  goal  that  have  not  been  discussed 


1  This  led  ultimately  to  including  options  like  TELLME  and  ALLDATA,  and 
incorporating  the  substantial  completion  termination  condition  for  a  topic  Section  6.2. 1.2. 


230 


DIALOGUE  MANAGEMENT  HEURISTICS 


(FOCUSRULES.RULEPK037).  The  candidates  are:  the  rule  that  was  thu  basis  for  a 
recent  hint,  rules  that  mention  a  particular  factor,  and  rules  that  support  a  particular 
hypothesis.  So  if  the  student  just  asked  how  to  USE  information  about  epidemics,  only 
the  rules  that  mention  epidemics  in  their  premise  and  have  not  yet  been  discussed  will 
be  considered  (these  are  called  the  rules  under  contention). 

The  second  step  is  to  choose  a  single  d-rule  from  the  list  of  rules  under 
contention.  GUIDON  applies  a  set  of  strategies  (RULEINTEREST.RULEPK.031 )  that 
assign  a  numerical  "interest"  score  to  each  d-rule;  the  list  is  sorted  on  this  basis  and 
the  top-ranking  one  is  presented  to  the  student  as  a  form  of  help  for  continuing  the 
diagnosis. 

The  t-rule  that  carries  the  most  weight  in  the  ordering  process  is  given  below 
(t-rule  27.01).  The  purpose  of  this  t-rule  is  to  capture  the  intuitive  notion  that,  in 
general,  change  In  belief  is  interesting:  the  more  drastic  the  change,  the  more 
interesting  the  effect.  The  numbers  In  the  conclusion  of  t-rule  27.01  are  certainty 
factors  that  indicate  our  belief  in  this  interestingness.  Thus,  the  domain  rule  that 
changes  the  current  hypothesis  most  drastically,  according  to  the  relative  weights 
given  by  this  rule,  will  be  selected  for  consideration  by  the  student  (other  factors 

being  equal).1 


1  Recall  that  the  value  of  a  goal  is  significant  if  its  CF  exceeds  the  cutoff 
(Section  5.1.5).  For  example,  if  the  goal  Is  the  "organism  causing  the  infection"  and 
the  certainty  associated  with  the  value  "pseudomonas"  is  .3,  then  this  value  is  not 
significant.  "Sum  over  the  computed  value"  means  that  the  effect  of  each  value 
concluded  by  the  rule  will  be  taken  into  account,  not  just  the  particular  value  that  will 
be  mentioned  if  the  chosen  presentation  method  is  to  state  a  hint. 
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T-RULE27 .0 1 


If:  1)  The  rule  succeeded  In  this  context,  and 

2)  The  following  value  has  been  determined:  a  hint  value  can  be 
suggested  for  this  rule,  based  on  its  effect  on  the  current 
hypothesis 

Then:  Modify  the  cumulative  belief  that  this  rule  should  be  presented 
by  the  sum  over  the  computed  value  as  follows: 

If  the  evidence  contributed  is  still  insignificant  then  .1 

If  a  new  Insignificant  value  is  contributed  then  .1 

If  "confirmation  of  related  evidence*  then  .3 

If  a  new  significant  value  is  contributed  then  .5 

If  a  new  strongly  significant  value  is  contributed  then  .75 

If  an  insignificant  value  becomes  significant  then  .B 

If  an  old  value  is  now  insignificant  then  .85 

If  belief  in  an  old  value  is  strongly  contradicted  then  .95 

Figure  6.8.  GUIDON'*  Strategy  for  Revising  an  Hypothesis 


T-rule  27.01  embodies  a  particular  strategy  for  revising  hypotheses.  When  the 
selected  d-rule  is  presented  to  the  student,  this  strategy  is  presented  indirectly  to 
the  student  as  a  form  of  motivation  for  considering  the  d-rule  at  that  point  in  the 
dialogue  (see  Figure  6.9  and  page  68  of  the  protocol).  GUIDON  essentially  implies,  "If 
you  knew  this  d-rule  and  were  following  this  strategy,  then  you  might  pursue  this 
evidence  at  this  time."  In  this  way,  the  program  would  be  underscoring  1 )  the  specific 
effect  of  the  d-rule  (when  the  d-rule  Is  introduced  as  one  that  strongly  confirms  the 
"bacterial"  hypothesis)  and  2)  the  strategy  of  seeking  evidence  that  strongly 
confirms  a  current  hypothesis  (though,  in  general,  t-rule27.01  is  designed  to  force 
discussion  of  evidence  that  overturns  a  current  hypothesis  if  possible). 


Of  course,  there  is  no  claim  that  following  the  strategy  of  t-rule  27.01  is  the 
best  way  to  do  diagnosis.  Instead,  the  reader  should  observe  that:  1 )  this  t-rule  is 
conceptually  a  meta-rule— a  strategy  for  applying  solution  methods,  2)  we  might  write 
other  rules  of  this  form  to  capture  other  strategies  for  revising  hypotheses,  end  3) 
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like  other  t-rules  this  is  an  heuristic  that  is  designed  for  experimentation.  For 
example,  another  strategy  would  emphasize  orderly  collection  of  basic  data  and 
forward-directed  inferences  to  focus  on  most  likely  hypotheses. 

Finally,  there  are  other  t-rules  for  ranking  the  rules  under  contention.  These 
take  into  account  the  student  model,  whether  the  rule  is  on  the  lesson  plan,  how  much 
work  remains  to  be  done  before  the  rule  can  be  applied,  and  whether  the  rule 
incorporates  tables  or  lists  in  its  premise.  In  the  current  version  of  GUIDON  these  are 
primarily  place-holders  that  illustrate  typical  factors  that  might  be  taken  into  account. 
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{ During  discussion  of  the  type  of  the  infection,  the  student  asks  for  assistance.} 

**  HELP 

Hint:  Look  for  strong,  positive  evidence  that  the  type  of  the 
infection  is  bacterial. 


{The  previous  request  for  help  set  some  global  variables  (the  * focus  record")  thot 
are  notu  used  when  the  tutor  provides  assistance.  Specifically,  GUIDON  selected 
d-rulebOl  (using  t-rule27.0l  and  others)  and  chose  to  give  a  hint  instead  of 
presenting  the  d-rule  ( based  on  the  student  model  and  features  of  the  d-ru't). 
Now,  with  the  second  request  for  help,  GUIDON  presents  the  rule  by  discussing  it 
step-by-step  (t-rulel6.04).} 

**  HELP 

[Applying  tutorial  rule  16.04] 

[Use-History  of  RULE601 :  0  ->  -400] 


{T he  student  model  is  altered  to  reflect  the  fact  that  the  student  didn’t  follow  up 
on  the  hint  by  considering  ruleiOl.  (It  is  not  clear  at  this  point  what  the 
modification  should  be.)} 

Let's  consider  a  rule  that  contributes  strong,  positive  evidence 
that  the  type  of  the  infection  is  bacterial.... 


(Now  considering  RULE501.) 


Figure  6.9.  Using  a  Problem-solving  Strategy  to  Construct  a  Hint 


234 


DIALOGUE  MANAGEMENT  HEURISTICS 


Helping  the  Student  to  Make  an  Hypothesis 

When  asked  to  make  an  hypothesis,  the  student  can  request  help.  GUIDON'S 
strategy  for  providing  help  is  to  select  a  rule  that  has  been  successfully  applied  by 
MYCIN  and  has  not  yet  been  discussed.  The  selection  and  presentation  methods  are 
the  same  as  those  described  above.  Though  of  course  since  the  selected  rule  has 
already  fired,  no  additional  case  data  will  be  presented  and  no  new  subgoals  will  be 
set  up  for  discussion;  a  question  is  usually  posed.  The  tutoring  principle  is  to 
encourage  the  student  to  consider  inferences  that  can  be  made  using  information  he 
has  already  been  given. 

A  Few  Complications 

There  Is  an  inherent  problem  in  trying  to  cleanly  separate  the  rule  selection 
strategies  from  the  rule  presentation  strategies.  In  particular,  recall  the  t-rule  that 
represents  GUIDON’S  strategy  for  revising  hypothesis  (t-rule27.01 ),  which  causes  a 
particular  d-rule  to  be  chosen  because  of  the  interesting  revision  it  brings  about.  The 
hypothesis  value  that  GUIDON  deems  most  interesting  at  this  time  should  be  the  same 
value  that  GUIDON  mentions  to  the  student  in  a  hint,  if  this  is  the  chosen  presentation 
method.  Therefore,  the  t-rule  that  states  the  hint  must  know  about  the  t-rule  for 
revising  hypotheses.  How  is  this  possible?  When  the  d-rules  are  ranked,  t-rule27.01 
records  with  each  rule  the  value  that  should  be  stated  In  a  hint. 

Interactions  of  this  kind  are  rare  in  GUIDON,  but  they  illustrate  the  general 
problem  of  control  when  the  decision  of  what  to  do  is  separated  from  its  execution. 
The  program  must  maintain  a  record  of  possible  and/or  intended  actions  on  a  sort  of 
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sketchpad  memory,  so  that  after  all  possibilities  have  been  weighed,  the  program  can 
return  to  the  best  plan  and  follow  it.  This  is  essentially  what  GUIDON  does  when 
constructing  a  hint. 

However,  the  problem  is  more  than  just  "remembering  what  you  decided  to  do." 
In  general,  it  is  possible  that  the  planning  phase  will  require  descriptions  of  the 
alternative  actions,  rather  than  independent  measures  of  what  is  desirable.  For 
example,  it  is  possible  that  a  good  reason  for  presenting  a  rule  is  that  the  situation 
allows  a  preferred  presentation  method  to  be  used.  Suppose  that  we  know  that  the 
student  prefers  for  GUIDON  to  begin  the  discussion  of  a  rule  (by  giving  case  data  or 
setting  up  a  subgoal),  rather  than  providing  a  hint.  The  t-rules  that  rank  the  d-rules 
under  contention  would  have  to  take  into  account  the  pros  and  cons  of  valid 
presentation  methods,  rather  than  simply  looking  at  the  student  knowledge  model,  or 
abstract  properties  of  the  rules.  Ideally,  the  ranking  t-rules  would  directly  examine 
the  presentation  t-rules,  or  perhaps  a  program-generated  description  of  them.  For  the 
moment,  we  must  hand-craft  interactions  between  t-rule  packets. 

6. 2.3.3  Opportunistic  Tutoring 

Opportunistic  teaching  strategies  are  intended  to  periodically  examine  the 
student's  understanding  and  introduce  new  information.  For  example,  a  dialogue 
transition  (from  GOAL.PROCOOI  to  RECENTSUCC.PROC043)  occurs  every  time 
exchange  of  case  data  causes  d-rules  to  succeed,  on  the  chance  that  the  student 
has  just  demonstrated  knowledge  of  a  d-rule  (by  requesting  case  data  relevant  only 
to  this  rule)  that  the  model  indicates  is  unknown  to  him.  Thus,  opportunistic  tutoring 
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interruptions  are  made  1)  in  an  appropriate  context  (as  part  of  a  discourse 
procedure),  and  2)  when  the  tutor  discerns  that  interruption  is  justified.  Tutoring 
interruptions  are  of  course  interesting  because  of  their  possible  beneficial  effect  on 
the  student's  learning.  Two  plausible  effects  are:  1 )  the  short-term  effect  of 
introducing  judgments  that  will  be  useful  when  the  student  expresses  his  hypothesis 
for  the  given  goal,  and  2)  long-term  exposure  to  more  rules  over  several  sessions,  and 
thus  better  retention  (in  this  sense  the  tutor's  interruptions  will  be  deliberate 
attempts  to  review  previously  discussed  material). 

While  it  is  not  too  hard  to  formalize  when  interruptions  should  be  considered,  it  is 
quite  difficult  to  formulate  good  strategies  for  deciding  when  such  an  option  should  be 
taken.  Preliminary  investigation  indicates  the  tutor  will  have  to  balance  the  cost  of 
interrupting  the  student  against  the  benefit  gained  from  improving  the  model  of  his 
understanding,  and  thus  being  in  a  better  position  to  offer  appropriate  remarks  later. 
For  example,  it  might  prove  advantageous  to  verify  the  student  model  right  away, 
rather  than  allowing  a  weak  hypothesis  of  what  the  student  knows  to  ripple  down  the 
line  and  pose  problems  later.  Opportunistic  tutoring  experiments  are  considered  at 
the  end  of  this  section. 

Rules  that  Recently  Succeeded 

GUIDON  has  two  strategies  for  exercising  the  option  to  present  a  d-rule  that 
just  succeeded.  The  first,  detecting  "unexpected  expertise,"  was  described  earlier 
(t-rule  43.02).  Based  on  the  student  model,  we  might  have  predicted  that  the 
student  would  not  request  the  data  used  by  a  rule.  By  interrupting  and  asking  a 
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question,  we  attempt  to  realign  the  model  with  the  student's  actual  understanding, 
perhaps  giving  the  student  a  rule  that  he  didn't  know  about  and  can  now  apply.  Note 
that  this  strategy  reflects  the  only  extent  to  which  GUIDON  follows  the  student's 
question-asking.  Moreover,  rather  than  trying  to  account  for  each  piece  of  information 
he  requests,  we  wait  until  the  rules  that  use  these  data  succeed.  This  elementary 
modeling  method  may  be  useful  to  Indicate  the  worth  of  constructing  a  much  more 
extensive,  and  expensive,  scheme. 


A  second,  significantly  more  complex  interruption  involves  discussing  an  entire 
subtree  of  the  solution  so  that  the  student  can  make  an  inference  (t-rule  43.01 ). 


T-RULE43 .81 


If:  1)  The  rule  has  not  been  discussed, 

2)  There  Is  not  strong  evidence  that  the  student  has  applied 
the  rule  under  consideration,  and 

3)  There  is  a  subgoal  that  the  student  probably  does  not  know 
Then:  Substep  i.  Print  a  design  of  a  line  of  dashes 

Substep  11.  Say:  intro-recentsucc-tree 

Substep  111.  For  each  of  the  rules  in  this  subtree  that  the 
student  probably  cannot  apply  because  he  is 
unable  to  determine  one  or  more  subgoals 
Do:  Present  missing  rules  to  the  student  so 
that  he  can  determine  the  missing  subgoal, 
then  quiz  about  rules  that  use  these 
subgoals  [Proc846] 

Substep  1v.  Print  a  design  of  a  line  of  dashes 
Figure  6.10.  Tutoring  Rule  to  Detect  and  Present  a  Missing  Subtree 
Here  we  make  a  distinction  betyveen  knowing  a  rule  and  being  able  to  apply  it  to  a 
particular  case.  For  example,  we  may  have  (a  priori)  reason  to  believe  that  the 
student  knows  that  "if  the  type  of  the  infection  is  fungal,  then  treat  for  cryptococcus 
or  coccidioides."  For  a  student  at  a  certain  level  of  sophistication,  this  rule  will  be 
obvious.  However,  determining  whether  or  not  the  type  of  infection  is  fungal  is  non- 
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trivial;  it  requires  knowing  several  specialized  rules  for  interpreting  cerebral-spinal 
fluid  tests.  The  purpose  of  t-rule  43.01  is  to  present  the  specialized  rules  in  a 
context  in  which  the  student  can  immediately  see  their  benefit  In  the  light  of  what  he 
already  knows.  Figure  6.11  graphically  illustrates  this  strategy. 


KNOWN-RULE 


[  known  subgoals  ]  UNKNOWN-SUBGOAL 


[  known  rules  ]  [  unknown  rules  ] 


- -  3.  Quiz 


» -  2.  Complete  goal 


»■  —  1.  Give  unknown 
rules  &  treat  known 
rules  recursively 


Figure  6.1 1.  ANQ/OR  Tree  Representation  of  the  Strategy  Used  by  T-rule43.01 


This  diagram  shows  the  discourse  pattern  followed  for  each  level  of  the  tree 
having  known  rules  with  unknown  subgoals,  l.e.,  subgoals  that  the  student  is  not  able 
to  sufficiently  determine  so  that  he  can  apply  the  known  rule.  The  presentation 
method  (UNKNOWNRULES.PROC046)  consists  of  presenting  d-rules  that  the  student 
doesn't  know  (step  1),  asking  him  to  make  an  hypothesis  about  the  subgoai  (step  2). 
and  then  quizzing  him  about  the  known  rule  that  uses  this  subgoal  (step  3).  If  there 
are  any  known  rules  (for  determining  the  unknown  subgoal)  with  unknown  subgoals, 
then  these  are  dealt  with  before  step  1,  in  a  recursive  manner.  The  dialogue  excerpt 
below  illustrates  this  procedure.  (In  this  example,  all  left- justified  output  in  brackets 
("[  ...  ]")  Is  trace  information  that  indicates  which  t-ruies  are  being  applied,  labels  of 
print  statements,  and  remarks  about  the  student  model.) 
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[T he  tutor  and  student  were  discussing  "the  infection  that  requires  therapy"  when 
the  procedure  for  discussing  a  goal  noticed  that  a  relevant  rule  (d-rule333)  had 
just  fired  ( based  on  evidence  given  to  the  student).  The  student  model  indicates 
that  the  student  should  know  this  rule  (if  is  a  simple  definition),  however  the 
model  also  indicates  that  he  can’t  apply  the  rule  because  it  mentions  a  subgoal  that 
he  doesn’t  know  how  to  determine  (the  infection  associated  with  the  CSF  culture). 
GUIDON  Invokes  the  opportunistic  tutoring  procedure  for  rules  that  succeed  in 
order  to  give  the  student  enough  information  so  that  he  can  apply  d-rule333 


TREAT INF 


0-RULE333  (recent  success  that  Is  a  definition 
|  and  presumed  to  be  obvious  to  the 

j  student) 

I 

INFECTION  (subgoal  used  by  d-rule333;  It  is  known 
|  by  HYCIN  at  this  point,  but  the  model 

1  Indicates  that  the  student  can’t 

j  deduce  It:  he  Is  "missing"  a  rule) 


D-RULE336  (GUIDON  assumes  that  the  student 

can’t  apply  this  rule  (even  though  It 
Is  a  definition)  because  It  uses 
a  list  that  associates  body  sites 
with  Infections.) 

GUIDON  will  present  d-rule336  to  the  student  and  then  encourage  him  to  make 
an  hypothesis  that  involves  using  d-rule333.  If  the  model  is  right,  information 
from  d-rule336  is  all  the  student  needs  to  wrap  up  the  current  goal 
(TREATINF).} 

[Applying  tutorial  rule  43.01] 


[INTRO-RECENTSUCC-TREE:] 

We  will  now  consider  evidence  that  leads  to  a  conclusion  about  the  infection 
that  requires  therapy. 

[INTRO-NEWVALS:] 

Here  are  some  conclusions  that  you  may  not  have  considered.... 

The  site  of  CULTURE' 1  is  evidence  that  the  Infection  with  which  this  pending 
csf  culture  Is  associated  Is  meningitis  [RULE336]. 

[Assuming  that  S  can  now  apply  RULE336  CULTURE- 1] 


* 


k 


1 


lanK* 
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[Reconsidering  S's  use  of  INFECTION  CULTURE-1  J 
[Applying  tutorial  rule  41.01  (else)] 

[Applying  tutorial  rule  41.02] 

[Student  applied  (RULE333  CULTURE- 1)  =  875] 
[S  can  probably  apply  RULE333  CULTURE-1  now.] 


{ T-rule  packet  41  updates  the  student  knowledge  model.  GUIDON  now  chooses  a 
question  format  (.randomly)  using  t-rule  packet  24....} 

[Applying  tutorial  rule  24.01] 

[Applying  tutorial  rule  24.02] 

[Applying  tutorial  rule  24.04] 

[Applying  tutorial  rule  24.05] 

[Applying  tutorial  rule  24.06] 

[Applying  tutorial  rule  24.09] 

[Premqtype  «  FACTS;  Actqtype  «  ACTVAL] 


[T he  question  will  state  the  rule  in  simple  true-false  form.} 

Does  the  infection  with  which  this  pending  csf  culture  is  associated  tell 
you  that  the  infection  that  requires  therapy  is  meningitis? 

««  YES 


[Use-History  of  RULE333:  0  ->  200] 
That's  right  (see  RULE333). 


{This  turned  out  to  he  a  trivial  question  because  it  is  a  "definition"  Nevertheless, 
the  point  is  made.  This  process  of  presenting  "missing"  rules  and  getting  the 
student  to  use  this  new  information  in  rules  he  already  knows  works  to  any  depth 
in  the  search  tree.  That  is,  GUIDON  will  track  down  and  present  all  of  the 
missing  rules  and  force  hypotheses  until  a  conclusion  can  he  drawn  about  the  topic 
that  caused  this  opportunistic  tutoring  procedure  to  be  invoked.} 


It  turns  out  that  the  tutor  doesn't  find  many  opportunities  to  apply  this  method  of 
presenting  information  to  the  student.  This  in  itself  is  interesting,  because  it  raises 
the  possibility  that  the  most  complex,  cleverest  tutoring  methods  will  be  so 
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specialized  that  the  cost  of  formalizing  and  testing  them  will  outweigh  the  benefits  of 
using  them  for  efficient  presentation  of  new  information.  On  the  other  hand,  it  is  a 
body  of  rules  like  t-rule43.01  that  lead  us  to  ascribe  intelligence  to  the  program. 
Evidently  we  are  impressed  by  the  specificity  of  the  sequence  of  goal-directed 
behavior. 

Aside  from  contributing  to  the  appearance  of  Intelligence,  opportunistic  tutoring 
rules  embody  a  particular  teaching  strategy  that  is  worth  considering  further:  they 
exemplify  the  active  component  of  teaching,  the  presence  of  mind  that  leads  a 
teacher  to  seize  the  initiative  to  improve  the  student's  understanding.  This  suggests 
a  demon-like  aspect  of  teaching,  In  which  t-rules  are  on  the  lookout  for  the  right 
moment  to  wake  up  and  guide  the  dialogue  by  stringing  together  just  the  right 
remarks,  gems  that  might  possibly  give  the  student  an  "Ah-ha*  feeling.  This 
perspective  also  suggests  the  pragmatic  aspect  of  teaching  that  is  inherent  in  the 
mixed-initiative  Interaction:  the  tutor  must  often  follow  the  student  and  wait  for  the 
right  moment  to  say  something,  rather  than  incessantly  prodding  him  along.  Perhaps 
the  validity  of  this  approach  is  revealed  by  the  success  of  the  SOPHIE  program,  in 
which  the  tutor  only  reacts  to  the  student's  initiative. 

To  recap,  using  GUIDON  as  a  model,  teaching  has  two  main  components:  listening 
to  a  student  in  order  to  understand  his  reasoning,  and  trying  to  control  him  by 
correcting,  restating,  and  demonstrating  problem-solving  methods  or  strnt»qj«>s. 
Opportunistic  tutoring  rules  tell  GUIDON  when  to  break  out  of  the  listening  phase  and 
attempt  to  control  the  student's  reasoning. 
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Quizzing  about  Rulas  that  Racantly  Faiiad 

As  the  student  acquires  case  data  for  a  goal,  many  of  the  rules  that  use  these 
data  will  be  found  to  fall.  While  GUIDON'S  general  strategy  is  to  discuss  the  rules  that 
succeed  when  the  student's  hypotheses  for  the  goal  are  discussed,  the  program  looks 
for  remarks  to  make  about  failed  rules  at  the  time  that  they  fail 
(RECENTFAILS.PROCOI  0).  In  order  to  avoid  a  prolonged  interruption,  the  program  ranks 
the  rules  that  just  failed,  using  the  "interest"  strategies  described  in  Section  6. 2.3. 2, 
discarding  rules  that  are  not  on  the  lesson  plan  or  unsuitable  for  discussion  (Appendix 
B  characterizes  "unsuitability").  Additional  case  data  are  presented  if  necessary,  and 
a  question  is  posed. 

Quizzing  about  "Related  Rules" 

At  various  points  In  the  dialogue,  after  a  given  d-rule  is  discussed,  GUIDON  looks 
for  related  d-rules  and  presents  them  to  the  student  (RELATEDRULES.PROCO1 1 ).  Two 
d-rules  are  related  If  they  mention  the  same  hypothesis  In  their  conclusion  and  have 
some  premise  subgoal  in  common.  Presenting  the  related  d-rule  currently  consists  of 
posing  a  question.  This  interruption  strategy  follows  the  general  principle  of  trying  to 
associate  new  Information  with  facts  the  student  already  knows.  In  particular,  the 
"related  rules"  strategy  takes  advantage  of  whatever  benefit  might  be  gained  from 
presenting  material  that  can  be  associated  with  subject  matter  that  the  student 
already  holds  in  mind.  (This  might  be  a  bad  idea;  maybe  connection  to  previously 
learned  material  is  more  effective  when  long-term  memory  associations  must  be 
reconsidered.) 
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Opportunistic  Tutoring  Experiments 

Modifying  opportunistic  tutoring  strategies  is  perhaps  the  most  likely  way  to 
experiment  with  GUIDON.  Experimentation  might  shed  fight  on: 

1 )  The  pedagogical  and  psychological  effectiveness  of 
alternate  presentation  and  quizzing  methods  (is  the  recursive 
approach  of  the  "unknown  subgoais/missing  rules"  strategy 
acceptable?), 


2)  Constraints  on  the  number  of  interruptions  in  a  given  time 
(similar  to  considerations  useful  in  the  WEST  program),  and 


3)  The  cost/ benefit  parameters  for  very  specific  methods 
that  are  infrequently  used  (and  hence  pose  system  overhead),  yet 
might  be  effective  because  of  their  directness. 


4)  The  affect  of  tutorial  remarks  on  student  Initiative  (do 
interruptions  encourage  or  inhibit  exploration?) 


6. 2.3.4  Tutoringi  Responding  to  a  Student's  Hypothesis 

At  any  point  in  the  dialogue  the  student  can  state  an  hypothesis  for  any 
subtask.  GUIDON  constructs  a  tree  that  relates  each  of  the  values  of  the  hypothesis 
(e.coli,  pseudomonas)  to  the  rules  that  MYCIN  used  to  form  its  own  hypothesis-  Only 
the  d-rules  that  MYCIN  has  been  able  to  apply  up  to  this  point  in  the  dialogue  are 
considered.  The  complications  of  the  modeling  process  are  described  in  Section 
8.3.2;  here  we  assume  that  the  values  have  been  grouped  according  to  their 
correctness,  and  rules  have  been  associated  with  each  value  according  to  whether 
the  rule  provides  evidence  for  the  value  or  provides  evidence  against  it.  The 
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strategy  of  the  "hypothesis  response"  t-rules  (HYPOTHESIS. PROC0 14)  is  to  quiz  the 
student  about  evidence  that  the  student  may  probably  know  or  may  have  forgotten. 
When  the  student  model  is  strong  (either  claiming  that  the  student  knows  a  rule  or 
does  not  know  it),  the  tutor  directly  states  evidence  without  quizzing. 

Evidence  is  classified  into  many  groups,  partly  on  the  basis  of  the  logic  of  the 
model,  and  partly  on  the  basis  of  distinctions  that  will  be  useful  for  tutoring.  These 
groups  are: 

1 )  correct  hypothesis  values 

a)  all  rules  that  MYCIN  used  are  probably  known  by  the 
student  (model  CF  >  .7). 

b)  some  rules  may  be  unknown  (  0  <*  model  CF  <=  .7). 

c)  some  rules  are  probably  unknown  (model  CF  <  0  ). 

2)  wrong  values  (hypotheses  MYCIN  explicitly  rejected) 

3)  missing  values  (significant  hypotheses  that  the  student  did  not  mention) 

4)  significant  rejected  values  (CF  <  -.2)  that  the  student  did  not  mention 

6)  Insignificant  values  (hypotheses  MYCIN  has  evidence  for,  but  does  not 
believe  [  -.2  <*  CF  <*  cutoff]) 

6)  values  that  are  not  concluded  at  all  by  the  rules  that  MYCIN  used 

7)  Insignificant  values  that  the  student  did  not  mention  (currently  ignored  by 
GUIDON) 
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GUIDON  deals  with  each  group  of  values  in  the  order  given.  The  chief 
presentation  method  (see  HYPOTHESIS. PROCOI 4,  HYPVALRESP0NSE.RULEPK025,  and 
PRESENTVALS.RULEPK026)  consists  of  asking  the  student  to  supply  a  list  of  factors 
that  support  a  specific  hypothesis,  be  it  correct,  wrong,  etc.  This  requires  special 
question  types  that  construct  a  question  using  a  list  of  rules  (Appendix  B).  Special  t- 
rules  single  out  cases  involving  a  single  (known  or  unkncwn)  d-rule  for  economical 
presentation;  for  example,  when  it  appears  that  the  student  made  a  wrong  hypothesis 
because  he  is  ignorant  of  a  single  rule,  the  program  confirms  the  evidence  that  the 
student  probably  knows  and  then  poses  a  direct  question  about  the  missing  rule  (see 
t-rule26.01). 

In  short,  GUIDON'S  response  to  an  hypothesis  is  an  attempt  to  be  logical  and 
thorough.  The  main  difficulty  we  have  observed  with  this  strategy  is  that  the  student 
Is  often  still  digesting  the  response  to  his  wrong  values,  when  GUIDON  is  already 
posing  questions  about  the  missing  values,  the  "no  evidence"  values,  and  so  on.  In 
the  protocol,  the  student  wanted  to  get  back  into  data  collection  mode  before  hearing 
more  about  his  hypothesis.  While  it  would  be  easy  enough  to  tell  the  program  to  skip 
the  rest  of  the  evaluation,  there  are  real  problems  if  the  student  wants  to  return  to 
hear  more  about  his  recent  hypothesis. 

The  problem  with  student  interruptions  during  the  hypothesis  evaluation 
procedure  is  that  many  of  the  internal  lists  set  up  during  the  modeling  phase  would 
need  to  be  updated  if  the  student  finds  out  more  about  the  case  between  questions. 
That  is,  if  another  rule  fires,  then  the  tree  of  evidence  that  supports  and  contradicts 
the  student's  hypothesis  will  be  out  of  date.  Since  this  tree  controls  which 
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"hypothesis  response"  t-rules  are  applied  and  is  used  to  generate  questions  about 
current  evidence,  the  tutor's  remarks  will  not  be  in  accord  with  the  current  knowledge 
model. 

It  is  of  course  possible  to  update  the  evidence  tree  (data  structure)  before 
returning  to  finish  hypothesis  evaluation,  but  the  problem  involves  more  than  time- 
consuming  computation.  In  general,  tutorial  decision-making  might  involve  carrying  out 
dynamic  plans  (sequences  of  presentations),  as  opposed  to  following  prepared 
procedures  that  rigorously  preserve  and  maintain  assumptions  about  the  discussion 
and  inference  records.  GUIDON'S  hypothesis  response  procedure  does  not  exactly 
plan  what  it  is  going  to  do,  but  its  modeling  stage  does  implicitly  enable  certain 
actions  that  are  to  follow.  Already  we  see  that  exchange  of  initiative  in  the  tutorial 
involves  complex  interactions  with  the  knowledge  models  and  discourse  procedures, 
given  that  student  interruptions  can  disrupt  a  plan  at  any  time.  It  appears  inevitable 
that  modeling  and  tutoring  will  have  to  go  on  in  tandem.  Specifically,  in  presentations 
of  any  considerable  length,  like  hypothesis  evaluation,  it  will  be  desirable  for  the  tutor 
to  periodically  stop,  listen,  and  reevaluate  its  present  course. 

6.2.4  Situation  Transition* 

Clarity  of  presentation  is  critical  In  productive  teaching,  as  in  any  dialogue,  in 
GUIDON  tutorials  this  involves  providing  coherent  transitions  between  dialogue 
situations  and  comprehensible  summaries  of  inferences. 

Providing  clear  transitions  requires  that  discourse  procedures  take  into  account 
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the  context  in  which  they  are  invoiced,  so  that  the  tutor's  remarks  are  not 
disconnected  or  redundant.  The  chief  design  principle  we  have  followed  is  that  a 
tutorial  rule  that  invokes  a  new  discourse  situation  provides  transition  to  the  new 
situation  and  return  to  the  former  context.  For  example,  T-RULE  5.03  prepares  the 
student  for  the  request  for  an  hypothesis.  Note  that  the  predicate  "an  introductory 
remark  is  to  be  made..."  is  a  flag  set  by  the  calling  discourse  procedure,  e.g.,  it  is 
false  if  the  student  has  explicitly  requested  the  opportunity  to  make  an  hypothesis. 

Figure  6.12  illustrates  the  kind  of  disconnected  dialogue  that  arises  from 
inadequate  communication  between  discourse  procedures.  Following  the  procedure 
that  is  invoked  after  an  hypothesis  ("advice"  in  Figure  6.3),  the  tutor  asks  the 
student  If  he  wants  to  continue  discussion  of  this  topic  or  if  he  would  like  to  wrap  it 
up  at  this  point.  The  student  wishes  to  continue,  so  the  dialogue  shifts  to  discussion 
of  this  topic  (the  procedure  GOAL),  and  the  first  thing  the  tutor  remarks  about  is  that 
the  topic  is  essentially  done  (attempting  to  make  an  economical  transition  based  on  a 
comparison  of  the  expert  program's  current  hypothesis  to  the  final  conclusion  it  will 
reach  after  all  evidence  has  been  considered). 
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**  HYPOTHESIS  38 

Is  the  meningitis  partially  treated? 

**  YES 

You’re  right. 

The  infection  which  requires  therapy,  the  fact  that  organisms  were  not  seen 
on  the  stain  of  the  pending  csf  culture  (CULTURE-1)  and  the  time  since 
therapy  with  the  cephalothin  ( DRUG- 1 )  was  started  is  evidence  that  the 
meningitis  is  partially  treated  (.95)  [RULE145]. 

It  remains  to  collect  further  evidence  for  determining  whether  the 
meningitis  is  partially  treated. 

Would  you  like  to  see  a  summary  of  all  of  the  evidence  that  can  ever,  ually 
be  collected  (as  opposed  to  working  it  out  yourself)? 

**  NO 

Would  you  like  to  see  a  summary  of  the  evidence  we  have  discussed  before 
going  on  to  ask  more  questions? 

**  NO 

We  have  substantially  completed  our  discussion  of  whether  the  meningitis  is 
partially  treated. 

Would  you  like  to  make  an  hypothesis  now  (as  opposed  to  gathering  more 
information)? 

**  NO 


Figure  6.12.  A  Disconnected  Dialogue 

A  smoother  dialogue  would  result  if  the  procedure  for  giving  advice  after  an 
hypothesis  looked  to  see  if  the  topic  was  substantially  completed  and  so  the  student 
would  only  be  asked  once  if  he  wanted  to  go  on.1  The  Interaction  shown  here  is 
typical,  and  many  of  the  changes  made  to  the  system  after  each  trial  tutorial  session 


1  This  change  has  been  incorporated  In  the  ADV1CEAFTERHYP.PR0C039 
procedure,  and  is  manifested  In  the  protocol  (page  56). 
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are  designed  to  improve  the  dialogue  in  similar  ways.  While  we  are  constantly 

for  more  subtle  variations  in  the  dialogue,  the  framework  of  rules  organized  into 

procedures  is  working  well. 

Finally,  when  tutorial  remarks  are  limited  to  lines  of  print  on  a  teletype  page  (a 
constraint  this  research  has  adopted),  there  is  no  opportunity  to  pause  or  to 
emphasize  words  (by  changing  voice  volume  or  inflection).  Therefore,  it  is  worthwhile 
to  incorporate  visual  keys  that  help  to  organize  presentations,  and  provide  clear 
transition  between  segments  of  the  dialogue.  For  example,  in  GUIDON,  we  make  use  of 
various  designs  that  are  intended  to  provide  familiar  transitions,  making  it  easier  for 
the  student  to  anticipate  what  will  be  expected  of  him  (e.g.,  the  use  of  ">>>"  before 
the  remark  that  the  expert  has  drawn  a  conclusion). 

When  summarizing  evidence,  GUIDON  regroups  goal/subgoal  relations,  separating 
key  factors  from  contextual  information.  Grouping  the  display  of  goal/subgoal 
information  can  make  the  large  amount  of  evidence  easier  to  read,  as  shown  on  page 
61.  Moreover,  only  the  key  factors  of  domain  rules  are  printed  here,  making  It  much 
easier  to  see  the  essential  evidence  (see  Section  6.2.4. 1  for  discussion  of  key 
factors).  Thus,  the  program  omits  clauses  that  are  common  to  the  set  of  rules  that 
are  being  mentioned,  stating  the  omitted  clauses  at  the  end,  as  a  reminder  of  the 

context  of  the  inferences.1  it  is  significant  that  the  usual,  MYCIN-style  listing  of  these 
rules  would  require  several  pages. 


1  Goldstein  [Goldstein,  1977]  has  considered  the  problem  of  simplifying  multiple 
inferences  to  clarify  arguments.  We  have  not  yet  found  this  to  be  necessary  for 
MYCIN-like  rule  bases. 
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'  6.3  Summary  of  Tutoring  Principles 

The  following  is  a  list  of  principles  or  general  strategies  used  in  GUIDON’S  t- 
rules.  Observe  that  a  given  principle  may  be  exhibited  in  many  individual  rules, 
allowing  for  the  various  situations  in  which  it  is  applied.  For  example,  there  are  more 
than  a  dozen  t-rules  for  discussing  a  domain  rule  economically,  based  on  what 
constitutes  economical  presentation  in  different  dialogue  situations  (number  of 
subgoals  left  to  discuss,  complexity  and  sophistication  of  the  rule).  In  general,  for  a 
given  tutorial  goal,  several,  sometimes  competing,  principles  must  be  considered  at 
once,  so  there  is  a  matrix  of  considerations,  rather  than  a  single  t-rule  for  each 
principle. 


1)  Be  perspicuous-,  economical  presentation  strategy  (Section 
6.2.1);  lucid  transitions  (Section  6.2.4);  adherence  to  conventional 
discourse  patterns  (Section  6.1. 3.1). 


2)  Account  for  incorrect  behavior  in  terms  of  missing 
expertise  (as  opposed  to  alternative  methods  and  strategies). 
Explain  clearly  what  is  improper  from  your  own  point  of  view.  (E.g., 
coping  with  Improper  requests  for  case  data,  Section  6. 2.2.1.)  (This 
is  of  course  more  a  statement  of  how  GUIDON  models  the  student, 
rather  than  a  principle  of  good  teaching.) 


3)  Strictly  guide  the  dialogue-,  say  when  topics  are  done  and 
inferences  are  completed,  as  opposed  to  letting  the  student 
discover  transitions  for  himself  (Section  6. 2.2. 2). 


4)  Provide  orientation  to  new  tasks  by  top-down 
refinement-.  Provide  the  student  with  an  organized  framework  of 
considerations  he  should  be  making,  without  giving  away  the  solution 
to  the  problem  (Important  factors,  subgoals,  size  of  the  task),  thus 
challenging  him  to  constructively  examine  his  understanding 
(Section  6.2.3.1). 
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6)  Provide  assist  anc e  by  methodically  introducing  small 
steps  that  will  contribute  to  the  problem's  solution  (Section  6. 2. 3. 2). 


1)  Assistance  should  at  first  be  general,  so 
as  to  remind  the  student  of  solution  methods  and 
strategies  he  already  knows. 


2)  Assistance  should  encourage  the  student 
to  advance  the  solution  by  using  case  data  he 
has  already  been  given. 


6)  Take  advantage  of  opportunities  to  examine  the  student's 
understanding  and  Introduce  new  Information  (Section  6. 2.3.3). 


7)  When  responding  to  partial  student  solutions,  probe  the 
student's  understanding  when  you  are  not  sure  what  he  knows, 
otherwise  directly  confirm  or  correct  the  solution  (Section  6.2.3. 4). 


6.4  Dialogue  Management  Conclusion 

While  most  ICAI  research  has  focused  on  representation  of  domain  expertise  and 
construction  of  a  student  model,  we  have  shown  that  there  is  a  group  of  issues  that 
center  about  the  problem  of  carrying  on  a  coherent,  task-oriented  mixed-initiative 
dialogue  with  a  student.  We  have  named  this  collection  of  issues  the  dialogue 
management  problem  and  discussed  it  In  terms  of  tutorials  based  on  MYCIN-like  rules. 
We  presented  a  representation  of  dialogue  knowledge  in  the  form  of  a  transition 
diagram  in  which  the  nodes  are  discourse  situations  and  the  links  represent  selection 
of  alternative  dialogues  based  on  domain  logic,  economy  of  presentation,  and  tutorial 
objectives.  Other  issues,  to  be  discussed  In  Chapter  7,  are  managing  topic 
relationships  and  sharing  Initiative. 
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We  stated  that  the  main  forcing  function  behind  improvement  of  the  program  was 
the  impossibility  of  explicitly  discussing  each  inference  made  by  the  expert  program. 
As  illustrated  in  Chapter  9,  verbosity  continues  to  be  a  problem  for  GUIOON.  It  is 
interesting  to  note  that  the  most  effective  pruning  of  topics  in  the  current  version  of 
the  program  is  a  result  of  the  student's  initiative.  Perhaps  it  is  not  necessary  or 
desirable  for  the  program  to  attempt  to  manage  the  dialogue  too  severely;  deciding 
what  should  be  discussed  is  naturally  a  shared  task.  We  believe  that  a  proposed 
case  lesson  plan  (Section  10.4.1)  will  complement  the  procedures  for  non* 
exhaustive  discussion  by  providing  reasonable,  time*sensitive  goals  for  the  session. 
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Chapter  7 

Providing  for  and  Coping  with  Student  Initiative 


CONTENTS 
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7.1.1  Setting  Case  Data 
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7. 1.5. 2  The  Focus  Record 
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7.4  Changing  the  topic:  OISCUSS,  TELLME,  control-F  interrupt 

7.5  Standard  user  options:  COMMENT,  PROFILE,  T,  OPTIONS 
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An  essential  part  of  tutorial  dialogue  management  is  allowing  the  student  to 
express  himself.  To  construct  a  mixed-initiative  program,  provision  must  be  made  for 
every  potential  kind  of  initiative  that  the  student  will  be  allowed  to  make.  This 
includes  being  able  to  refer  back  to  an  early  topic  and  provide  more  details  (given 
that  explicit  presentation  of  evidence  may  be  non-exhaustive),  change  the  topic,  and 
so  on.  We  might  summarize  this  by  saying  that  we  must  allow  the  student  to  specify: 
what  he  knows,  what  he  wants  to  know  more  about,  and  what  he  wants  to  ignore. 

In  addition,  a  tutor  must  cope  with  the  initiative  it  recognizes.  This  means  that 
the  tutor  should,  to  some  degree,  respect  student  initiative  by  patterning  its  own 
actions  according  to  the  lines  of  thought  taken  up  by  the  student.  For  example,  when 
assistance  is  provided,  It  should  reflect  what  the  student  has  done  so  far  to  solve  the 
problem.  Responsive  tutoring  requires  continuous  monitoring  and  record-keeping  about 
the  context  of  the  dialogue. 

GUIDON  provides  for  student  initiative  in  two  related  ways.  First,  a  wide  variety 
of  options  are  available  to  the  student,  and  second,  many  tutorial  remarks  are  indexed 
so  that  the  student  can  easily  refer  to  them  later  (use  them  as  arguments  to  options). 
To  relate  its  own  remarks  to  the  student's,  the  tutor  maintains  simple  records  of  the 
context  of  the  dialogue,  consisting  of  subgoals,  hypothesis  values,  and  inferences 
that  were  recently  mentioned. 

The  options  available  to  a  student  are  summarized  in  a  table  that  he  can  print 
during  the  tutorial  (listed  in  Figure  7.1).  As  described  below,  each  option  generally 
has  a  discourse  procedure  associated  with  It.  For  example,  the  procedure  for  IKNOW 
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invokes  HYPOTHESIS.PROCOI  4  if  the  expert  program  hasn't  made  a  final  decision  yet; 
otherwise  COMPLETEDGOAL.PROCOOO  is  invoked.  Simple  English  phrases  are  also 
accepted  by  the  program  ("How  can  I  conclude  that  the  infection  is  viral?"). 
However,  we  believe  that,  for  the  student  community  with  which  we  are  working,  this 
option  format  (often  called  menus)  will  prove  to  be  more  convenient. 

Option  type  Summary  of  options 


Get  Case  Data 

DATA 

BLOCK  ALLDATA 

Information  Retrieval 

FACTORS 

USE 

SUBGOALS  PENDING 

CONCLUDE  DETAILS 

QA  SUMMARIZE 

Dialogue  Context 

RULE 

TOPIC  ORIENT 

Convey  What  You  Know 

IK  NOW 

HYPOTHESIS 

Request  Assistance 

HELP 

HINT  TELLME  QUIZ 

ALLRULES 

Change  the  Topic 

DISCUSS 

STOP  control-G 

Special  ?  PTR  PR  JUSTIFY  LITERATURE 

DEFINE  COMMENT  PROFILE  OPTIONS 

[Key:  PR  s  print  domain  rule;  PTR  s  print  tutorial  rule; 
control-G  =  give  initiative  to  the  student;  QA  =  enter  MYCIN’s 
question-answer  module;  f  «  "what  Is  valid  here7" 

See  Appendix  C.] 

Figure  7.1.  Options  Available  During  GUIDON  Dialogues 

The  discussion  that  follows  is  organized  according  to  these  option  groups.  In 
Section  7.1  options  for  getting  case  data,  retrieving  information,  and  establishing 
the  current  context  are  related  under  the  topic  of  sharing  and  maintaining  problem¬ 
solving  context:  what  is  known  so  far?,  what  remains  to  be  done?,  what  are  we  doing 
now?  Subsequent  sections  discuss  options  by  which  the  student  conveys  what  he 
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knows,  requests  assistance,  and  changes  the  topic.  Finally,  we  will  consider  some 
standard  options  that  make  the  program  easier  to  use. 

7.1  Sharing  and  Maintaining  Problem- Solving  Context 

The  problem-solving  context  is  represented  in  GUIDON  by  the  AND/OR  solution 
tree.  The  nodes  are  annotated  as  the  dialogue  proceeds  to  keep  track  of  what  has 
been  mentioned  to  the  student  and  what  part  of  the  tree  has  been  completed  by  the 
expert  program.  Clearly,  the  overlay  model  is  part  of  the  record  of  problem-solving 
context;  but  because  of  its  complexity  we  will  consider  It  separately  in  the  next 
chapter.  The  options  considered  here  are  designed  to  provide  structured  information 
to  the  student.  The  Instructional  goal  is  to  provide  windows  on  the  solution  that  will 
encourage  the  student  to  explore  his  own  understanding  by  raising  curiosity  about  the 
expert's  approach. 

7.1.1  Getting  Case  Data 

7. 1 .1 . 1  Options:  DATA,  BLOCK,  and  ALLDATA 

The  student  can  request  case  data  by  either  giving  the  keyword  name  of  the 
parameter,  optionally  prefaced  by  the  command  DATA.  The  "context,"  e.g., 
ORGANISM-1,  is  optional  when  it  Is  unique.  Blocks  of  data,  a  group  of  related  tests, 
can  also  be  requested.  "BLOCK  7"  will  print  all  recognized  block  names.  Note  that  the 
procedure  for  supplying  case  data  (GIVENPARMDATA.PROC012)  will  automatically 
provide  all  related  tests,  once  one  is  requested.  Similarly,  when  a  screening 
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parameter  (e.g.,  surgery)  is  requested,  the  data  it  screens  (neurosurgery)  is  provided 
as  well.  Finally,  the  student  can  ask  GUIDON  to  print  all  of  the  case  data  that  MYCIN 
requested  (for  the  current  goal)  that  hasn't  been  given  to  him  yet  by  using  the 
ALLDATA  option. 

Section  8.3.2  describes  how  the  overlay  model  is  updated  when  case  data 
are  given  to  the  student.  Below  we  consider  rhetorical  patterns  for  refocusing  the 
dialogue  when  a  data  request  violates  the  goal-directed  nature  of  the  dialogue. 

7. 1.1. 2  Focusing  for  Data  Bequests 

Student  requests  for  data  can  Implicitly  change  the  topic  if  the  datum  requested 
Is  not  relevant  to  the  current  topic,  if  the  datum  cannot  be  used  directly  in  any 
inference,  that  is,  it  is  not  a  factor  of  the  current  goal,  then  it  is  not  "relevant"  to  the 
current  goal.  If  the  question  is  unnecessary,  the  student  is  told  why  (as  described  in 
Section  6.2.2. 1).  Otherwise,  the  datum  must  be  a  factor  of  a  deeper  subgoal  or  a 
shallower  goal,  page  69  illustrates  what  occurs  when  the  requested  information  is 
relevant  to  a  deeper  subgoal.1  If  the  requested  information  is  relevant  to  a  previous 
and  shallower  goal,  the  tutor  states  this  relation  so  that  it  is  clear  to  the  student  what 
topic  is  currently  being  pursued  (Figure  7.2). 

In  making  these  distinctions  and  connections  for  the  student,  the  program 
maintains  shared  focus  on  the  current  goal  while  respecting  the  student's  specific 
interest  in  other  aspects  of  the  case.  It  is  here  that  the  goal-directedness  of  the 


1  If  this  is  not  unique,  the  program  chooses  one  arbitrarily. 
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dialogue  is  enforced:  though  note  that  the  student  can  explicitly  change  the  topic  at 
any  time  by  the  DISCUSS  option. 


earlier  goal 
related  subgoal 


current  topic 


question 


REGIMEN 


CROWD 


**  DOES  THE  PATIENT  LIVE  IN  A  CROWDED  ENVIRONMENT? 

Pt538  does  not  live  In  a  crowded  environment. 

Whether  the  patient  does  live  in  a  crowded  environment  is  not 
relevant  to  determining  the  type  of  the  Infection.  It  is  a  consideration  we 
can  use  later  when  we  return  to  our  discussion  of  the  organisms  (other  than 
those  seen  on  cultures  or  smears)  which  might  be  causing  the  Infection. 

Figure  7.2.  Crossing  Paths  in  a  Tree  of  Subgoals 


7.1.2  Inference  and  Oiscussion  Records 

The  inference  and  discussion  records  consist  of  marks  placed  on  the  AND/OR 
solution  tree.  The  inference  record,  which  indicates  which  rules  have  been  applied  by 
the  expert  program  and  the  conclusions  made  by  these  rules,  is  a  component  of  the 
overlay  model;  it  is  described  in  Section  8. 3. 2.2.  The  discussion  record 
indicates  which  (case-specific)  inferences  have  been  mentioned  in  the  dialogue  and 
which  goals  have  been  completed  (the  tutor  has  mentioned  MYCIN's  final  conclusion). 

This  record  Is  also  useful  to  the  student  who  may  find  it  difficult  to  keep  track 
of  the  evidence  that  has  already  been  discussed  and  what  remains  to  be  considered: 
two  critical  aspects  of  dialogue  management.  Without  an  accurate  record  of  the 
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status  of  the  dialogue,  it  would  be  difficult  for  the  participants  to  reach  a  consistent, 
final  agreement  about  the  topics  under  discussion.  Figure  7.3  illustrates  one  of  the 
several  options  by  which  GUIDON  shares  its  record  with  the  student. 


**  CONCLUDE  VIRAL 

Using  the  following  factors,  it  will  be  possible  to  conclude  that  the  type 
of  the  infection  Is  viral: 

10a.  the  duration  of  the  neurological  symptoms  [RULE526] 

18b.  the  white  count  from  Pt538' s  peripheral  CBC  (in  thousands)  and 
the  WBC  from  the  CSF  C RULE 504  ] 


Do  you  want  to  see  contrary  evidence? 

**  YES 

Using  the  following  factors,  it  is  possible  to  conclude  that  the  typn  of  the 
infection  is  not  viral: 

11a.  the  WBC  from  the  CSF  and  the  percent  of  PMNs  in  the  CSf  (RULE 501] 


Do  you  want  information  about  evidence  that  we  have  already  discussed  (if 
any)? 

**  YES 

Using  the  following  factors,  we  have  already  discussed  how  to  conclude  that 
the  type  of  the  Infection  is  viral: 

12a.  the  duration  of  the  neurological  signs  [ RULE 524 1 


Do  you  want  to  see  contrary  evidence? 
**  NO 


Figure  7,3.  Keeping  Track  of  Evidence  for  an  Hypothesis 


In  this  example  of  the  CONCLUDE  option,  observe  that  the  program  refers  to 
incomplete  Inferences  (RULE526),  new  evidence  (RUIE501),  and  evidence  that  has 
already  been  discussed  (RULES24).  Some  tutorial  strategies  are  designed  to  directly 
offer  Information  like  this  to  the  student.  For  example,  after  a  rule  is  discussed 
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explicitly,  GUIDON  states  the  cumulative  hypothesis  that  has  been  discussed  for  that 

goal/  Examples  of  related  information  retrieval  options  are  given  in  Section 
7.1.4. 

7.1.3  Controlling  Tutorial  Remarks 

Another  student-controlled  means  of  sharing  status  information  is  through  the 
student  profile.  At  the  start  of  a  tutorial  session  the  student  specifies  whether  he 
wishes  to  be  told  about  conclusions  being  made  by  the  expert  program  (MYCIN)  in  the 
course  of  the  dialogue.  He  can  request  nothing,  ask  for  full  information,  or  simply 
request  a  "hint,"  the  suggested  usage.  See  protocol  page  43. 

The  components  of  the  profile  are: 

Student  sophistication 


The  student's  estimation  of  his  experience  is  matched  against  the  a  priori 
sophistication  ratings  for  each  rule.  See  Section  8.2.1. 


Failure  or  Success  of  0-ruies 


A  trace  of  domain  rule  applications  during  the  tutorial  provides  a  detailed  window 
on  each  decision  made  by  MYCIN.  It  Is  of  much  less  value  than  the  more  abstract 
descriptions  given  by  other  profile  options. 


Rule  Conclusions 


Successful  rule  conclusions  will  be  stated  directly,  held  until  the  relevant  topic 
enters  the  discussion,  described  in  hint  form,  or  suppressed.  A  hint  simply  states  that 
a  conclusion  can  be  made,  as  shown  in  this  excerpt: 


1  However,  GUIDON  does  not  model  the  student's  forgetting  during  the  session, 
so  this  information  is  given  to  every  student  every  time. 
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The  duration  of  the  neurological  signs  is  7.2  hours. 

>>>  37a.  MYCIN  just  made  a  conclusion  about  the  type  of  the  infection. 


In  terms  of  teaching  strategy,  this  kind  of  remark  is  intended  to  encourage  the 
student  to  examine  his  own  understanding  at  this  point,  perhaps  leading  him  to 
request  more  information.1 


Final  Goal  Conclusions 


Final  conclusions  for  a  goal  will  be  stated  directly,  described  in  hint  form,  or 
suppressed,  in  many  cases,  this  window  on  the  expert  program  is  less  useful  than  the 
individual  rule  conclusions  which  can  be  associated  with  specific  case  data  and 
completion  of  intermediate  subgoals,  as  illustrated  above. 


Lesson  Plan 


The  default  lesson  plan  will  tend  to  restrict  tutoring  (quizzes)  to  rules  that  are 
within  two  levels  of  the  student's  sophistication  rating.  This  default  provides  a  crude 
approximation  of  the  selective  tutoring  goals  that  the  lesson  plan  is  ultimately 
intended  to  represent. 


Debugging 


Debugging  information  includes  a  trace  of  t-rule  application,  as  well  the  setting 
of  some  important  modeling  and  focus  record  variables.  This  option  is  of  course  not 
intended  for  students. 


On  the  basis  of  limited  experience,  we  feel  that  hints  about  rule  and  goal 
conclusions  are  not  Intrusive  because  the  expert  program  is  only  infrequently  able  to 
apply  a  domain  rule  successfully  during  the  dialogue.  Two  heuristics  tend  to  prevent 


1  "DETAILS  37A' 
7.1. 4.3. 


will  open  up  discussion  of  the  relevant  rule;  see  Section 
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redundant  or  useless  remarks  of  this  kind.  First,  a  final  conclusion  is  not  mentioned  if 
no  rules  were  applied  (this  prevents  the  tutor  from  saying  that  MYCIN  has  made  a  final 
conclusion  about  case  data  immediately  after  the  value  is  given  to  the  student). 
Second,  intermediate  rule  conclusions  are  not  mentioned  if  only  one  rule  succeeds  for 
this  goal  (this  prevents  the  tutor  from  saying  that  MYCIN  has  made  a  conclusion  and 
then  saying  right  away  that  MYCIN  has  made  a  final  conclusion  for  the  same 
parameter). 

7.1.4  Direct  Retrieval  of  Domain  Knowledge 

Here  we  consider  the  most  basic  options  for  exploring  the  AND/OR  solution  tree. 
A  variety  of  options  provide  different  perspectives  on  what  has  been  accomplished  in 
the  task  and  what  remains  to  be  done.  All  of  the  options  are  interpreted  in  the 
context  of  the  goal  being  discussed.  Most  of  these  options  print  remarks  that  are 
indexed  for  later  reference  by  the  student.  Remark  indices  are  discussed  in  the 
section  about  the  DETAILS  option. 

7. 1 .4. 1  FACTORS,  SUBGOALS,  and  PENDING 

The  FACTORS  option  lists  all  parameters  that  appear  In  rules  for  the  current  goal. 
(This  is  the  union  of  CONSEQ-SG!,  FAIL-SGI,  and  ANTE-SG!,  with  duplicates  removed.) 
Elements  are  starred  (*)  if  they  have  been  discussed.  Entries  that  have  not  been 
discussed,  but  for  which  MYCIN  has  made  a  final  conclusion,  are  mentioned  at  the 
bottom  of  the  listing.  This  feature  is  intended  to  prompt  the  student  to  use  the 
DISCUSS,  TELLME,  IKNOW,  or  HYPOTHESIS  options  to  finish  up  these  subgoals. 
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The  SUBGOALS  option  prints  the  subtree  with  the  current  goal  at  the  root. 
Subgoals  are  grouped  (by  Indentation)  under  the  goal  to  which  they  are  relevant. 
Parameters  that  are  determined  by  case  data  alone  are  not  mentioned,  only  those 
factors  that  are  determined  by  one  or  more  rules. 

The  PENDING  option  prints  only  those  FACTORS  that  have  not  been  discussed. 
The  listing  is  in  three  parts:  subgoals  that  remain  to  be  discussed,  case  data  that 
remain  to  be  requested,  and  subgoals  for  which  the  student  has  enough  information  to 
form  a  final  hypothesis. 

All  three  of  these  options  take  as  an  optional  argument  the  name  of  a  qnal  «.>*•■ 
that  this  information  would  not  be  available  if  MYCIN  had  not  completely  worked  the 
case  before  the  tutorial  began.  By  reconfiguring  MYCIN's  rules  into  the  explicit  tree 
representation,  it  is  computationally  trivial  to  display  goal  and  data  relationships.  If 
GUIDON  did  not  have  access  to  the  final  Inference  structure,  it  would  only  be  possible 
to  say  what  had  been  determined  so  far  (starred  FACTORS).  The  tutor  could  look 
ahead  as  MYCIN  does  in  a  consultation  when  asked  HOW  some  Incomplete  goal  will  be 
achieved  [Davis,  1976],  but  this  might  be  slow  and  would  not  provide  the  map-like 
perspective  that  can  be  directly  derived  from  a  complete  solution  trace.  On  the  other 
hand,  human  tutors  don't  always  proceed  from  a  worked-out  solution,  so  we  certainly 
would  not  insist  that  GUIDON'S  style  is  the  best:  It  is  the  advantage  of  being  able  to 
lay  down  an  an  exhaustive  itinerary  of  points  to  explore  that  is  of  interest. 
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7. 1.4.2  USE  &  CONCLUDE 

The  USE  and  CONCLUDE  options  have  evolved  directly  from  their  MYCIN 
question-answer  program  equivalents:  How  do  you  use  a  piece  of  information?  How  do 
you  make  a  conclusion?  Rules  that  use  a  factor  are  determined  by  finding  the  subset 
of  successful  rules  (UPDATE-SUCC!  of  the  current  goal)  that  require  the  factor 
(SUCC-PARMS!  of  the  rule).  Note  that  GUIDON  indicates  the  conclusions  that  will  be 
made  using  this  factor,  and  does  not  refer  to  failed  inferences.  This  is  the  information 
that  was  deemed  to  be  most  useful  to  the  student.1 

Rules  that  conclude  a  value  are  determined  by  again  examining  the  successful 
rules,  in  this  case  looking  at  the  CONCLUSIONS  property  of  the  rules.  Again,  failed 
inferences  are  not  mentioned.  As  discussed  earlier  (Section  7.1.2),  GUIDON  separates 
discussed  conclusions  from  partially  completed  inferences  and  those  that  are  known 
to  MYCIN  but  have  not  yet  been  discussed.  In  addition,  evidence  that  contradicts  this 
hypothesis  value  is  printed  if  the  student  is  interested. 

7. 1.4.3  DETAILS 

Remark  indices  are  recorded  in  a  list  that  Is  indexed  by  the  number  of  the 
remark  and  labelled  by  one  of  a  dozen  remark  kinds,  e.g.,  a  subgoal,  or  a  rule  that  is 

mentioned  in  a  summary  of  evidence  for  a  topic.2  They  allow  easy,  unambiguous 


1  Complications  arise  when  no  rules  succeed,  when  many  rules  succeed  (the  rule 
model  is  printed  instead),  and  when  the  goal  rule  is  involved  (leading  to  giving  away 
the  case  solution). 

2  The  list  Is  called  INTERACTION.HISTORY.  Elements  are  of  the  form: 
^interaction  no.>  <remark  label>  {optional  arguments)  ). 
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references,  greatly  contributing  to  the  flow  of  information  between  tutor  and  student. 
A  remark  index  that  stands  for  a  goal  can  be  used  for  any  option  that  takes  a  goal  as 

an  argument,  as  can  be  seen  throughout  the  protocol.1 


The  procedure  for  providing  DETAILS  knows  how  to  provide  details  for  any 
remark  kind.2  Here  we  list  each  kind  of  remark,  Indicate  when  the  tutor  prints  it,  and 
summarize  the  details  provided. 

LISTS:  A  domain  list  that  appears  in  rules  for  a  goal. 

Printed  by:  the  FACTORS  option. 

Arguments:  name  of  the  list,  the  goal 

Details:  definition  of  the  list,  example  elements  of  the  list, 
values  concluded  by  rules  that  mention  the  list. 

FACTORS:  Parameter  of  rules  for  a  goal. 

Printed  by:  FACTORS  option. 

Arguments:  the  factor 

Details:  definition  of  the  factor,  how  the  factor  Is  USEd. 

FIRED:  Rules  that  have  been  applied  by  NYCIN,  but  possibly  not  discussed. 
Printed  by:  Summary  of  evidence  for  a  goal,  and  CONCLUDE  option. 
Arguments:  the  rule 

Details:  For  each  rule  factor  that  has  not  been  discussed,  offer 
the  student  a  summary  of  evidence  for  determining  the 
factor.  Offer  to  print  the  rule. 

PENDINGRULE:  Rule  that  has  not  been  applied. 

Printed  by:  CONCLUDE  option. 

Arguments:  the  rule 

Details:  offer  to  discuss  the  rule  clauss-byclause 

SUBGOALS:  Subgoal  for  a  goal. 

Printed  by:  SUBGOALS  option,  sketch  of  tree. 

■*  Arguments:  the  subgoal,  the  goal. 

Details:  Offer  to  show  how  the  subgoal  is  USEd.  Offer  a  choice 
of  discussing  the  subgoal  In  detail  or  seeing  the  final 
conclusion  and  summary  of  evidence. 

FINALCONCL:  Final  conclusion  made  by  NYC  IN 
Printed  by:  profile  option 
Arguments:  the  goal 

Details:  If  this  Is  the  current  topic,  suggest  that  the  student 

use  the  TELLHE  option  if  he  really  wants  to  end  discussion. 


1  If  necessary,  the  procedure  for  converting  remark  Indices  to  option  arguments 
converts  the  history  information  to  the  proper  form. 

2  This  was  originally  implemented  as  regular  INTERLISP  code,  but  we  find  now 
that  it  can  be  easily  converted  to  the  format  of  discourse  procedures. 
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If  the  goal  was  narked  as  discussed  before  MYCIN  had  a 
chance  to  apply  all  of  the  rules,  offer  to  discuss  them 
(uses  NOTDISCUBRULES . PROC00  7 ) . 

If  the  goal  has  already  been  discussed  normally,  express 
surprise  and  display  the  final  conclusion  and  offer  a 
summary  of  evidence  (FINISHGOAL.PROC006). 

Otherwise,  offer  a  choice  of  seeing  the  final  conclusion 
or  discussing  the  goal  in  detail. 

RULECONCL :  Hint  or  description  of  a  rule  that  succeeded. 

Printed  by:  Profile  option 

Arguments:  the  rule,  the  goal 

Details:  Print  the  key  factors  and  conclusion. 


7. 1.4.4  QA 

MYCIN’s  question-answer  facility  (Section  4.3.1)  is  accessible  from  GUIDON.  All 
of  the  standard  options  are  available.  The  main  reason  for  using  the  QA  is  to  ask 
general  questions  about  the  rule  base  (How  does  MYCIN  conclude  whether  an 
organism  is  a  contaminant?  What  organisms  are  normally  found  in  the  mouth?).  The 
only  other  feature  of  the  QA  not  generally  provided  by  GUIDON  is  that  failed 
inferences  are  routinely  mentioned  (GUIDON  prints  them  when  it  has  nothing  else  to 
say). 


7. 1.4.6  SUMMARIZE 

The  final  information  retrieval  option,  SUMMARIZE,  prints  the  evidence  that 
MYCIN  used  to  determine  a  goal.  Key  factors  and  conclusions  of  each  rule  are  printed. 
(This  might  be  contrasted  with  the  summary  method  used  in  hypothesis  evaluation 
which  groups  the  evidence  by  hypothesis  value.)  GUIDON'S  summaries  are  better  than 
a  list  of  rules,  but  could  be  improved  if  the  program  abstracted  the  most  important 
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evidence,  especially  by  describing  basic  lines  of  reasoning  in  the  rules.  For  example, 
when  summarizing  the  final  diagnosis,  the  program  might  say,  "Organism*  1  will  not  be 
treated  because  it  Is  a  contaminant,"  thus  delving  into  the  subtree  and  picking  out  a 
key  factor  from  the  rule  that  tipped  the  scale.  Summarizing  negative  decisions  could 
involve  more  than  listing  half-a-dozen  weak  factors.  To  some  degree  GUIDON'S 
description  of  evidence  could  be  improved  by  simply  substituting  prosaic 
characterizations  for  the  certainty  factors,  e.g.,  "weak  evidence"  or  "confirming 
evidence." 

7.1.5  Dialogue  Context 

Previous  sections  have  dealt  with  sharing  context  about  the  problem  being 
solved  (domain  facts,  what  remains  to  be  done).  Here  we  will  treat  the  topic  history 
of  the  dialogue  as  another  context  to  be  shared  (motivating  the  current  topic,  relating 
assistance  to  recent  topics). 

7.1 .6.1  Options t  RULE,  TOPIC,  and  ORIENT 

The  student  can  directly  request  a  description  of  the  current  topic  by  the  TOPIC 
option.  The  goal  that  is  being  pursued  at  that  time  will  be  printed,  with  a  list  of  typical 
hypotheses  that  might  be  considered.  The  hypothesis  list  is  sometimes  the  best 
description  of  the  current  goal.  "We  are  discussing  the  type  of  the  infection"  may  be 
vague  to  a  student:  how  does  this  differ  from  the  name  of  the  infection?  The 
additional  remark,  "You  should  try  to  determine  if  the  type  of  the  infection  is  bacterial, 
viral,  fungal  or  tb”  Is  a  better  statement  of  the  current  goal  (page  67). 
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The  RULE  option  will  state  the  fule  number  that  is  being  considered  This 
information  is  available  to  the  program  by  rebinding  the  variable  CURULE  when  a 
discourse  procedure  begins  discussion  of  a  rule,  e  g.,  this  is  done  in  RULE.PROC004 
and  SUGGESTRULE.PR0C031 . 

The  ORIENT  option  prints  the  stack  of  goals  and  d-rules  that  are  being  pursued 
(page  48).  Discourse  procedure  macros  (DISCUSS\\,  STRATEGY\\)  record  their 
arguments  on  a  history  stack.  A  special  procedure  reads  this  stack,  extracting  calls 
to  GOAL.PROCOOI ,  RULE.PROC004.  and  active  t-rules  (indicated  by  TRULE  in  the 
stack).  Finally,  the  deepest  (most  recent)  t-rule  is  retrieved  from  the  source  file  and 
steps  of  the  rule's  action  that  remain  to  be  performed  are  printed,  thus  demonstrating 

the  capability  to  look  ahead  and  say  what  will  happen  next.1 

The  current  implementation  of  the  ORIENT  feature  will  probably  be  of  little  value 
to  a  student.  Instead,  It  was  implemented  to  demonstrate  the  possibility  of  having 
GUIDON  answer  questions  about  its  teaching  strategies  during  the  tutorial.  That  is, 
the  well-structured,  compiled  code  will  make  it  possible  to  implement  a  HOW  and  WHY 
explanation  system,  as  in  the  MYCIN  consultation  system.  The  reasons  for  wanting 
GUIDON  to  be  able  to  explain  Its  reasoning  are  the  same  as  for  any  other  knowledge- 
based  program:  for  a  human  to  modify  the  program  he  must  understand  the  expertise 
that  the  program  is  currently  using  as  a  basis  for  its  decisions  [Barr,  Bennett,  & 
Clancey,  1979],  Thus,  the  ORIENT  option  demonstrates  the  feasibility  of  developing 


1  Each  STEP\\  and  SUBSTEP\\  of  a  discourse  procedure  marks  the  history  tree. 
Like  all  entries  in  the  tree,  this  mark  is  removed  (popped)  when  the  operation  is 
complete.  Thus,  GUIDON  finds  the  current  step  number  in  the  tree,  e  g..  (TRULE  48. 04 
(SS  iv.  (D!  GOAL.PROCOOI  (MUMPSYM  PATIENT-638)))  indicates  that  step  4  of  t- 
rule48.04  is  being  followed  and  the  current  topic  is  "symptoms  of  mumps." 
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an  explanation  and  knowledge  acquisition  package  for  interactively  modifying  t-ruies. 
It  may  also  be  useful  for  the  tutor  to  be  able  to  explain  to  the  student  why  a  certain 
topic  is  being  pursued  in  the  dialogue. 

7. 1.6.2  The  Focus  Record 

GUIDON  keeps  a  simple  record  of  topics  in  which  the  student  seems  to  be 
interested,  and  uses  this  record  when  taking  the  initiative  to  direct  the  dialogue.  The 
idea  is  to  adapt,  at  least  in  a  simple  way,  the  tutor's  choice  of  topics  to  those  the 
student  may  want  to  know  more  about.  The  focus  record  consists  of  variables  that 
are  set  when  the  student  uses  various  options,  it  is  used  by  the  tutor  to  select  a 
domain  rule  to  discuss  when  the  student  explicitly  requests  help  or  a  hint.  For 
example,  the  USE  option  mentions  a  topic  (subgoal  to  the  current  topic)  and  the 
CONCLUDE  option  mentions  an  hypothesis  (for  the  current  topic).  These  can  be  used 
to  index  the  rules  for  the  current  topic. 

In  addition,  GUIDON  sometimes  has  to  keep  track  of  its  own  remarks  so  that  it 
does  not  repeat  itsolf,  and  can  follow  up  properly.  For  example,  the  program  provides 
a  HINT  by  describing  the  hypothesis  that  is  supported  by  a  rule  that  it  would  like  the 
student  to  consider.  When  the  student  next  requests  HELP  or  another  HINT,  we  don't 
want  the  program  to  repeat  this  hint,  but  instead  want  it  to  do  something  else  with  the 
chosen  rule.  A  simple  variable  records  that  a  rule  has  been  mentioned  to  the  student 
in  the  form  of  a  hint,  and  tutorial  rules  for  providing  help  look  for  this  situation  in  order 
to  follow-up  appropriately  (Figure  6.9).  The  means  for  keeping  the  dialogue  focused 
are  admittedly  simple,  but  because  of  the  other  constraints  on  the  dialogue  (i.e., 
stacked  topics  and  the  option-oriented  format),  they  seem  to  be  sufficient. 
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7.2  Conveying  What  You  Know:  (KNOW  &  HYPOTHESIS 

A  student  may  directly  offer  an  hypothesis  for  any  goal  at  any  time  by  using  the 
hypothesis  option.  It  is  interesting  to  note  that  GUIDON  uses  MYCIN's  consultation 
code  to  state  a  question  and  parse  the  values  and  certainty  factors  of  the  answer. 
The  actual  response  to  the  hypothesis  is  given  by  the  procedure 
HYPOTHESIS.PROCOI  4,  as  described  in  Section  6.2.3  4. 

The  IKNOW  option  means  "I  don't  think  that  we  have  to  discuss  this  goal  in  detail 
because  I've  already  figured  out  a  solution."  The  rule  packet 
IKNOWOPTION.RULEPK044  responds  according  to  the  status  of  the  goal:  whether  it 
has  been  discussed,  whether  It  is  unrequested  case  data,  and  whether  or  not  MYCIN 
has  reached  a  final  conclusion.  If  MYCIN  is  not  done,  the  student  is  requested  to 
state  his  hypothesis  (t-rule  44.03).  Otherwise,  the  procedure 
COMPLETEDGOAL.PROCOQ6  will  use  the  student  model  to  decide  whether  the  student's 
hypothesis  should  be  discussed. 

We  believe  that  these  strategies  Illustrate  one  of  the  strengths  of  the  GUIDON 
t-rule  framework.  Given  a  rich  context  (the  student  says  he  knows  something),  a  set 
of  very  specific  t-rules  can  be  written  to  detect  alternative  situations  and  then  cause 
the  tutor  to  act  In  a  very  precise,  and  hence  apparently  intelligent  way.  The 
framework  Is  convenient  because  it  focuses  our  attention  (as  teacher-experts)  on  a 
particular  dialogue  situation  (signified  by  the  rule  packet  and  conditions  for  its 
invocation),  and  asks  us  to  consider  how  we  would  act  in  this  situation.  Production 
rules  appear  to  be  particularly  suitable  for  encoding  this  kind  of  "what  would  you  do 
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when"  knowledge.  T-rules  provide  e  language  for  stating  tutorial  dialogue  actions: 
start  up  another  dialogue  pattern,  say  something,  or  make  a  record. 

7.3  Requesting  Assistance:  HELP/HINT,  QUIZ,  TELLME,  ALLRULES 

The  procedure  for  providing  help  (HELPFORGOAL.PROC016)  is  described  in 
Section  6.2.3. 2.  Quizzes  are  generated  by  ranking  the  rules  that  have  been  applied 
by  MYCIN,  using  the  same  "interest"  strategy  as  for  providing  help.  A  question  is  then 
generated  from  the  rule  (Appendix  B).  If  the  student  requested  a  hint,  the  "most 
Interesting  value"  (Section  6.2.3.2)  Is  stated  as  the  hypothesis  to  be  pursued. 

The  ALLRULES  option  directs  the  tutor  to  discuss  the  rules  for  the  current  topic. 
Subgoals  will  be  set  up  and  discussed  in  the  normal  way.  This  option  leads  to  an 
overly  long  dialogue  if  there  are  many  rules  to  be  discussed  (say,  more  than  5). 

If  the  student  finds  himself  short  of  time,  bored,  or  hopelessly  unable  to  work  out 
a  subproblem,  the  TELLME  option  will  provide  an  immediate  end  to  the  topic,  with  the 
final  conclusion  and  a  summary  of  evidence  provided  upon  request.  Like  the 
procedure  for  dealing  with  the  IKNOW  option,  TELLMEOPTION.PROC045  makes  a  number 
of  distinctions  based  on  the  status  of  the  topic  the  student  would  like  to  "close  out." 
The  TELLME  option  is  particularly  useful  to  get  past  some  of  MYCIN's  intermediate 
decisions  that  are  of  little  tutorial  value,  for  example,  the  step  that  combines  the  list 
of  hypothesized  unknown  organisms  from  cultures  with  the  list  of  organisms  that  will 
be  treated  based  on  clinical  evidence  alone  (page  82  and  following). 


1  If  he  wishes  to  terminate  the  session  itself,  he  can  type  STOP. 
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7.4  Changing  the  Topic;  DISCUSS,  TELLME,  control-F  interrupt 

The  student  can  direct  the  tutor  to  open  discussion  of  a  topic  by  the  DISCUSS 
option.  The  procedure  PLEASEDISCUSS.PR0C022  manages  the  transition  between  old 
and  new  topics,  by  commenting  on  the  relation  of  the  new  topic  to  the  old,  printing  a 
design  to  mark  off  the  shift  in  focus,  and  by,  as  usual,  explicitly  saying  which 
transitions  are  taking  place. 

The  TELLME  option  Is  included  in  this  list  because  it  might  also  be  used  to 
change  the  topic.  Thus  it  might  mean  "let's  move  on,  I've  had  enough"  as  well  as 
"help  me  get  out  of  this  situation,  I  can't  deal  with  It  today." 

By  typing  a  control  Interrupt  (control-F)  the  student  can  signify  that  he  would 
like  to  say  something.  Currently,  this  interrupt  character  causes  a  flag  to  be  set  that 
is  examined  after  the  tutor  poses  a  question.  This  is  a  particularly  good  time  to  check 
to  see  if  the  student  has  a  question,  especially  since  the  program  may  have  embarked 
on  a  long  sequence  of  posing  questions,  as  when  It  responds  to  the  student's 
hypothesis  for  a  subproblem.  If  the  flag  is  set,  the  student  Is  prompted  for  input,  but 
his  options  are  restricted  to  data  retrieval  and  special  options,  like  making  a  comment 
or  changing  his  profile.  The  problem  of  letting  him  request  case  data  or  change  the 
subject  during  an  interrupt  is  discussed  in  Section  B.2.3.4. 

7.6  Standard  User  Options;  COMMENT,  PROFILE,  7,  OPTIONS 

Options  that  we  now  consider  to  be  standard  for  a  program  like  GUIDON  Include 
the  ability  to  file  a  comment  about  the  program,  to  change  the  user  profile,  to  get  bn, 
for  using  an  option  (text,  plus  sample  arguments),  and  to  list  valid  options. 
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In  displaying  valid  options  we  have  been  influenced  by  the  work  of  Nelson 
([Nelson,  1074],  pp.  DM49-50).  We  group  the  options  according  to  the  kind  of 
operation  the  student  may  have  in  mind  (saying  that  he  knows  something,  changing  the 
topic,  getting  case  data,  retrieving  MYCIN  facts)  (see  Figure  7.1). 
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In  any  dialogue,  it  is  useful  for  the  speaker  to  have  a  model  of  what  his  listener 
knows,  is  thinking  about,  or  would  like  to  know  more  about.  This  is  particularly  true  ir 
a  tutorial  dialogue,  in  which  the  tutor's  purpose  is  to  be  informative.  In  order  to  know 
what  is  new  to  the  student,  and  so  worth  saying,  the  tutor  will  have  to  analyze  the 
student’s  behavior  for  clues  about  what  he  knows.  This  is  the  problem  of  assigning 
credit  to  domain  facts,  relations  and  strategies  to  explain  the  student’s  behavior.  We 
say,  "he  is  asking  about  the  chest  x-ray,  perhaps  he  is  considering  pneumonia  "  Or, 
"he  says  that  the  organism  is  e.coli,  he  must  not  have  considered  the  gram  stain."  A 
second  problem  is  to  revise  the  model  on  the  basis  of  new  evidence  For  example,  if 
the  tutor  tells  the  student  the  correct  answer  to  a  question,  should  it  expect  him  to 
get  the  question  right  next  time?  On  a  more  practical  level,  simple  record  keeping  of 
what  has  been  discussed  can  be  difficult  when  the  dialogue  jumps  about  or  does  not 
treat  each  subproblem  in  exhaustive  detail. 

Before  we  can  consider  the  maintenance  of  the  student  model,  its  growth  and 
revision,  we  must  consider  just  what  the  model  should  contain:  what  aspects  of 
behavior  are  we  seeking  to  explain?  This  question  is  directly  tied  to  what  we  will 
want  to  do  with  the  model,  and  how  it  is  represented.  The  final  sections  of  this 
chapter  review  uses  of  the  student  model  that  are  described  elsewhere,  and  consider 
the  adequacy  of  MYCIN-like  rules  as  a  representation  for  modeling  the  student's 
behavior. 

8.1  Contents  of  the  Model 


We  generally  think  of  a  student  model  as  an  evolving  description  of  what  the 
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student  knows  about  the  subject  domain.  From  this  perspective,  we  start  with  a 
corpus  of  facts  and  relations,  say  rules  or  a  semantic  net,  and  "paint  in  red"  what  we 
think  the  student  knows.  As  described  below,  this  dynamic  process  is  based  on 
various  sources  of  evidence  and  ways  to  reason  about  the  evidence,  involving 
considerations  like  measures  of  a  priori  difficulty,  structural  relationships  in  the 
knowledge  base,  and  common  sense  reasoning. 

However,  from  another  perspective,  it  is  too  simplistic  to  talk  about  the  student 
model  as  if  it  Is  a  description  or  list  of  facts  contained  in  the  student's  head.  We  can 
get  a  better  grasp  on  what  the  model  should  contain  if  we  think  of  it  in  terms  of  what 
the  student  is  doing,  his  cognitive  state  [Winograd,  1977].  A  cognitive  student 
model  characterizes  domain  knowledge  in  terms  of  its  purpose  and  application  to 
problem  solving.  We  might  say  that  the  student's  attention  is  focused  on  a  certain 
subgoal  (determining  the  type  of  the  Infection),  he  is  considering  a  particular  method 
(analyzing  CSF  test  findings),  and  that  he  is  following  a  certain  strategy  (differential 
diagnosis).  From  the  perspective  of  the  cognitive  process,  we  will  naturally  broaden 
our  model  to  include  process  distinctions  like  whether  the  student  is  proceeding  on 
the  basis  of  what  he  knows,  or  whether  he  is  following  up  on  lines  of  reasoning 
suggested  by  the  program.  For  example,  this  aspect  of  the  model  could  be  very 
useful  for  deciding  whether  to  interrupt  or  what  kind  of  suggestion  to  make. 

In  short,  the  tutor  is  modeling  a  thinking  person;  the  cognitive  model  will  be  used 
to  explain  what  the  student  is  trying  to  do.  A  tutorial,  like  any  Interaction  in  a  dialogue, 
can  be  described  as  a  "shared  process  of  trying  to  know."  This  requires  that  the 
participant  models  embrace  more  than  a  list  of  "known"  facts.  Given  this  criterion  for 


278 


THE  STUDENT  MOOEl 


the  student  model,  we  use  MYCIN's  representation  of  knowledge  to  characterize  the 
model's  contents  In  terms  of  rules  and  strategies  for  doing  diagnosis. 

We  consider  a  rule  to  be  a  skill  or  problem-solving  method,  and  make  a 
distinction  among  whether  the  student  knows  the  method,  is  able  to  use  it  (can 
achieve  prerequisite  subgoals),  and  has  actually  applied  it  to  a  particular  problem. 
Knowledge  of  skills  has  "local"  value  for  explaining  the  student's  requests  for  case 
data  (a  given  method  requires  specific  data  to  be  applied),  and  explaining  student 
hypotheses  about  subproblems  (a  given  method  contributes  to  the  solution  in  a 
specific  way).  Skills  are  an  Important  part  of  what  the  tutor  wants  to  teach:  does  the 
student  ask  for  relevant  data  and  draw  appropriate  conclusions? 

Strategies  serve  to  select  problem-solving  methods.  We  make  a  distinction 
between  domain-specific  strategies  and  task-specific  strategies.  A  domain-specific 
strategy  orders  methods  on  the  basis  of  domain  knowledge.  A  task-specific  strategy 
is  an  approach  for  attacking  the  problem,  e.g.,  hypothesis  revision  strategies  and  data 
collection  protocols.  Unlike  problem-solving  skills,  strategies  have  a  global  effect  and 
may  be  followed  concurrently.  Indeed,  there  is  reason  to  believe  that  people  need  to 
follow  meta-strategles  for  selecting  strategies.  However,  we  will  not  consider  meta¬ 
strategies  here  (see  Section  6.3.3  for  related  discussion). 

An  important  consideration  In  tutoring  is  the  use  by  the  student  of  rules  and 
strategies  that  do  not  correspond  to  "expert"  knowledge  already  represented  in  the 
program.  However,  the  technique  of  overlay  modeling,  used  by  GUIDON  (introduced  in 
Section  4.4.2),  attempts  to  explain  the  student's  behavior  exclusively  in  terms  of 
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MYCIN's  rules  and  strategies  (see  Figure  8.1).  This  means  that  alternative,  justifiable 
problem-solving  behavior  might  be  incorrectly  diagnosed  as  wrong,  and  misconceptions 
might  be  detected,  but  never  understood  by  GUIDON  (so  the  program  wouldn't  be  able 
to  explain  to  the  student  what  is  wrong).  Even  when  the  student  reaches  the  same 
conclusions  as  MYCIN,  we  have  no  guarantee  that  he  Is  using  MYCIN's  rule  set;  there 
may  be  other  reasoning  paths  baaed  on  a  logically  adequate,  but  different  set  of 


rules. 


6UIDQN  assumes  student 
knowledge  Is  a  subset 
of  expert  knowledge  and 
that  there  are  unique 
reasoning  paths. 


Figure  8.1.  Assumptions  of  GUIDON'S  Overlay  Model 

Finally,  note  that  the  current  version  of  GUIDON  does  not  model  student 
strategies.  First,  MYCIN  does  not  have  a  well-developed  set  of  problem-solving 
strategies  (metarules),  Indeed  there  are  only  two  metarules  In  the  system.  Protracted 
sessions  with  expert  physicians  may  be  necessary  in  order  to  formalize  a  useful  set 
of  such  strategies.  Second,  considerable  experimentation  involving  analysis  of 
protocols  will  be  required  before  we  can  say  what  the  task-specific  strategies  are 
that  we  should  be  trying  to  detect.  Once  strategies  have  been  formalized  and 
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represented  so  that  GUIOON  can  reason  with  them,  a  later  version  of  the  program 
could  incorporate  them  in  the  student  model. 

In  spite  of  these  limitations,  MYCIN's  production  rules  constitute  a  complex  and 
comprehensive  body  of  knowledge,  easily  justifying  their  use  as  the  foundation  of  an 
overlay  model.  Creating  the  model  itself  from  consultation  records  is  sufficiently 
difficult  to  warrant  study.  Pushing  the  program  to  its  limits  In  trial  dialogues  can  be 
expected  to  provide  valuable  experience  for  developing  models  of  alternative  or 
flawed  reasoning  methods  and  strategies.  The  final  section  of  this  chapter  (Section 
8.5)  shows  how  such  a  study  might  proceed. 

8.2  Sources  of  Evidence 

Following  Carr  and  Goldstein  [Carr  &  Goldstein,  1977],  we  classify  evidence  for 
growing  and  revising  the  student  model  into  three  groups:  background  evidence  for 
initializing  the  model,  implicit  evidence,  and  explicit  evidence. 

8.2.1  Background  Evidence 

When  GUIOON  wants  to  know  if  the  student  knows  a  domain  rule  (in  order  to 
follow  some  tutorial  rule),  it  checks  for  evidence  collected  In  previous  tutorial 
sessions  with  this  student  (USE-HISTORY  property  of  the  d-rule).  If  the  rule  has 
never  been  considered  before,  then  a  t-rule  packet  (BACKGROUND.RUIEPK040)  is 
applied  to  initialize  the  model.  The  current  set  of  t-rules  illustrates  typical 
background  considerations.  However,  it  has  not  been  tested  extensively,  and  may 
omit  some  useful  relations.  The  background  t-rules  are: 
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1)  The  rule  sophistication  ranking  is  compared  to  the 
student's  estimation  of  his  problem-solving  experience  (Section 
7.1.3).  If  the  student's  level  is  greater  than  the  sophistication 
level  of  the  rule,  then  this  is  strong  evidence  that  the  student 
knows  the  rule  (for  example,  If  the  rule  were  stated  as  a  quiz,  he 
would  answer  the  question  correctly),  if  the  levels  are  the  same, 
then  this  is  considered  to  be  only  moderate  evidence.  Rule 
sophistication  levels  are  normative:  they  represent  what  a  student 
at  a  certain  level  should  be  expected  to  know.  These  levels  were 
determined  Infdrmally  by  the  rule  authors,  based  on  their  experience 
and  the  source  for  the  rules.  For  example,  esoteric  rules  derived 
from  medical  Journal  articles  were  given  the  "specialist"  (4  on  a 
scale  from  0  to  4)  ranking. 


2)  A  definitional  rule  (one  that  concludes  a  value  with 
certainty)  is  considered  to  be  easier  to  learn  than  one  that  is  based 
on  uncertain  evidence.  If  the  definitional  rule  is  one  that  the 
student  has  probably  seen  before  (at  or  below  his  sophistication 
level),  then  this  strongly  increases  the  tutor's  belief  that  the 
student  knows  this  rule.  However,  If  the  definitional  rule  references 
a  domain  list  (e.g.,  the  association  between  body  sites  and 
infection),  then  this  weakens  our  belief  that  the  student  will  know 
how  the  rule  applies  to  a  new  situation,  so  we  don't  modify  the 
student  model. 


3)  Rules  with  a  clause  that  involves  iteration  seem  to  be 
difficult  to  learn.  This  is  considered  to  be  strong  evidence  (again  a 
priori)  that  the  student  will  not  be  considering  this  rule  during  the 
tutorial  (and  would  have  trouble  answering  a  question  about  it). 

Background  considerations  that  GUIDON  does  not  explicitly  take  into  account 
include  relations  among  the  rules:  analogies,  generalization-specialization  links,  and 
prerequisite  ties.  Some  of  these  connections  are  no  doubt  implicitly  recorded  in  the 
assigned  rule  sophistication  ranking.  For  example,  rules  that  are  duals  (Section  6.1.4) 
have  the  same  sophistication  level.  Nevertheless,  if  we  have  evidence  about  one  rule 
from  tutorial  sessions,  this  is  probably  a  stronger  basis  for  initializing  the  student 
model  for  the  dual  than  the  default  ranking. 
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8.2.2  Implicit  Evidence 

Indirect  clues  can  be  used  to  reason  about  the  knowledge  guiding  the  student's 
behavior.  In  GUIDON  this  Implicit  evidence  can  be  derived  from  the  student's 
hypotheses  and  his  choice  of  options.  The  current  version  of  GUIDON  doesn't  analyze 
the  student's  choice  of  options,  but  it  is  clear  that  the  data  he  requests,  the  forms  of 
assistance  he  requires,  and  even  the  remark  indices  he  refers  to  might  be  used  to 
deduce  the  student’s  focus  of  attention,  whether  he  is  struggling  to  solve  the 
problem,  his  domain  problem-solving  strategies,  and  so  on.  As  a  rudimentary  form  of 
this  aspect  of  modeling,  when  the  student  requests  help  for  solving  the  problem  (HELP 
or  HINT  option),  the  program  lowers  its  belief  that  the  student  knows  the  rule  that  is 
selected  for  presentation. 

Undoubtedly  the  most  important  form  of  implicit  evidence  in  a  GUIDON  tutorial  is  a 
student's  hypothesis  for  a  subproblem.  The  program  attempts  to  account  for  a 
student's  hypothesis  in  terms  of  d-ruies.  The  reasoning  involved  in  growing  the  model 
from  this  evidence  is  described  In  Section  S.3.2.6.  An  important  problem 
involves  apportioning  credit  or  blame  when  multiple  knowledge  sources  could  account 
for  the  observed  student  behavior  and  information  about  intermediate  student 
reasoning  steps  Is  missing. 

8.2.3  Explicit  Evidence 

A  tutor  can  also  find  out  what  the  student  knows  or  Is  thinking  about  by  asking 
him  directly.  GUIDON  constructs  questions  from  one  or  more  rules,  and  uses  the 
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student's  answer  as  explicit  evidence  about  what  he  knows.  (We  do  not  ask  what 
the  student  Is  doing,  because  we  do  not  attempt  to  model  this.)  Appendix  B  details 
the  process  of  constructing  questions  from  rules,  and  evaluating  and  responding  to 
the  student's  response.  The  paradoxical  problem  of  revising  the  model  after  a 
question  is  asked  is  considered  in  Section  8.3.3. 

8.3  Growth  and  Revision  of  the  Modal 

Here  we  describe  how  GUIDON'S  overlay  model  Is  maintained  during  a  single 
tutorial  session.  In  contrast  to  the  larger  perspective  on  the  modeling  problem  that 
we  have  considered  so  far  (strategies,  focus  of  attention,  cognitive  style),  this 
section  considers  the  problem  of  modeling  the  student's  knowledge  solely  in  terms  of 
MYCIN's  d-rules.  In  particular,  we  describe  the  components  of  the  model  and  show 
how  the  program  maintains  this  model  by  reasoning  from  the  available  background, 
implicit,  and  explicit  evidence. 

8.3.1  The  Components  of  the  Model 

The  components  of  the  student  model  consist  of  properties  of  d-rules  and 
properties  of  problem-solving  goals.  During  the  course  of  a  tutorial,  three  properties 
are  associated  with  each  d-rule: 

1)  USE-HISTORY:  The  only  component  that  persists  from 
session  to  session,  this  represents  the  tutor's  belief  that  the 
student  knows  the  rule.  That  Is,  he  is  likely  to  get  the  right  answer 
If  he  is  quizzed  about  the  rule.  When  a  tutorial  rule  refers  to  "the 
student  model,”  it  is  accessing  USE-HISTORY. 


284 


THE  STUDENT  MODEL 


2)  SAPPLIED?:  This  is  the  tutor's  belief  that  the  student  has 
applied  the  d-rule  during  this  session.  That  is,  he  has  been  able  to 
determine  the  evidence  required  by  the  rule's  premise,  and  is  aware 
of  the  conclusions  made  by  the  rule's  action.  He  is  likely  to  use 
these  conclusions  in  hypotheses. 


3)  USED?:  This  is  the  tutor's  belief  that  the  student  has  used 
the  rule  to  form  the  hypothesis  (that  he  has  just  stated).  That  is,  if 
asked  to  support  his  hypothesis,  he  would  refer  to  this  rule. 

The  following  diagram  shows  how  these  components  are  related  to  each  other  and  to 

the  sources  of  evidence  for  modifying  the  model. 


update  when  a  update  during 

d-rule  fires  hypothesis  evaluation 


Background 


USE-HISTORY  *=»»*»>>  SAPPLIED?  =«===»  USED? 


Assistance  Quiz 
Needs 


Hypothesis 


Figure  8.2.  Maintenance  Relations  for  Model  Components 


This  diagram  Illustrates  the  distinction  that  GUIDON  makes  among  having  used  a 
rule  in  the  past  (USE-HISTORY),  being  ablt  to  use  it  in  a  particular  case  (SAPPUFI>~  l 
and  actually  using  it  (USED?).  All  three  components  have  certainty  factors  associated 
with  them.  Three  ranges  of  belief  are  distinguished:  strong,  uncertain  (between 
strong  and  O)  and  negative  (less  than  0).  "Strong  belief”  Is  a  threshold  determined 
by  a  global  variable,  STRONGCF,  that  is  used  throughout  the  program  (currently  .7). 

On  the  left  of  this  diagram  we  see  the  sources  of  evidence  for  the  student 
model.  The  line  from  USED?  to  USE-HISTORY  illustrates  the  feedback  nature  of  the 
modeling  process.  Evidence  that  the  student  knows  a  rule  (USE-HISTORY)  contributes 


Growth  and  Revision  of  the  Model 


285 


to  evidence  that  he  has  applied  the  rule  (SAPPLIED?),  which  contributes  to  evidence 
that  he  actually  used  the  rule  to  make  an  hypothesis.  Finally,  the  hypothesis 
evaluation  itself  provides  Implicit  evidence  that  the  student  knows  the  rule  (feedback 
to  USE-HISTORY). 

Relatively  complex  reasoning  is  required  to  keep  the  three  belief  components  of 
the  model  up  to  date.  Maintenance  of  the  basic  overlay  model  (SAPPLIED?), 
evaluation  of  an  hypothesis  (USED?),  and  revision  of  the  cumulative  model  (USE- 
HISTORY)  are  described  next. 

8,3.2  The  Overlay  Model 

As  the  student  is  given  case  data  during  the  tutorial,  d-rules  that  MYCIN  used  in 
the  corresponding  consultation  are  Invoked  in  a  forward  direction.  In  this  wnv. « .  ,  ><  »N 
knows  at  every  moment  what  the  expert  program  would  conclude  based  on  the 
evidence  available  to  the  student.  Special  t-rules  (SAPPLIED.RULEPK041 )  determine 
whether  the  student  is  reaching  the  same  conclusions.  This  section  describes  the 
basic  operation  of  the  overlay  model,  end  then  details  a  wealth  of  complications  that 
are  illustrated  by  the  example  protocol. 

8.3.2. 1  Example  of  Modeling  Process 

The  following  Is  an  excerpt  from  the  protocol  (starting  on  page  55),  showing  the 
modeling  process.  The  following  subtree  of  MYCIN's  solution  is  relevant: 
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TYPE  =  bacterial 

Rule  ^48  ^ules  569,  591,  592,  etc. 

/“A  .. 

Partially  Treated  ■  True 


Rule  145 

/A... 


Treat  Infection  *  Meningitis 


The  example  Illustrates  when  the  components  of  the  model  are  updated. 
Observe,  in  particular,  the  problem  of  accurately  determining  what  the  student  knows 
at  each  step  along  the  way. 


»>  16a.  MYCIN  has  made  a  FINAL  CONCLUSION  about  the  infection  which 
requires  therapy. 


{After  determining  the  infection  that  requires  therapy,  MYCIN  was  able  to  apply 
Rulel45.  T -rules  then  show  that  the  student  does  not  know  this  rule  (based  on 
background  information )  and  cannot  apply  it  (it  is  assumed  that  he  cannot 
perform  the  mapping  operation  clause  3  requires).} 

[Applying  tutorial  rule  41.01] 

BACKGROUND.RULEPK040--(RULE1 45  CULTURE-1 ) 

[Applying  tutorial  rule  40.03] 

[Backgrounds  of  RULE  145  «  -800  ] 

[SModel  update  for  RULE  145:  NIL  ->  -900] 

[S  can’t  apply  RULE  145  CULTURE-1  due  to  clause  3] 

[Applying  tutorial  rule  41.04] 

[Student  applied  (RULE  145  CULTURE- 1)  ■  -1000] 

»>  1 7a.  MYCIN  has  made  a  FINAL  CONCLUSION  about  whether  the  meningitis  is 
partially  treated. 
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{MYCIN  applied  rulel4S,  using  the  information  about  the  meningitis  being 
partially  treated.  Since  the  student  does  not  know  this  information,  he  cannot 
apply  the  rule.} 

[Applying  tutorial  rule  41.01] 

BACKGROUND.RULEPK040— (RULE1 48  CULTURE- 1 ) 

[Backgrounds  of  RULE  148  *  0] 

[SModel  update  for  RULE  148:  NIL  ->  0] 

[S  can't  apply  RULE  148  CULTURE- 1  due  to  clause  2] 

[Applying  tutorial  rule  4 1 .04] 

[Student  applied  (RULE  148  CULTURE- 1)  •  -1000  ] 

>>>  18a.  MYCIN  just  made  a  conclusion  about  the  type  of  the  infection. 


[Later,  the  student  states  his  hypothesis  about  the  type  of  the  infection....} 

What  is  the  type  of  the  infection? 

"»  BACTERIAL  (9) 


[T -rules  compare  this  value  to  the  rules  applied  by  MYCIN.  It  so  happens  that 
rulel48,  which  it  is  believed  the  student  did  not  apply,  is  evidence  for  the 
hypothesis  of  a  bacterial  infection....} 

[Applying  tutorial  rule  1 4.04  (else)] 

RULEUSED.RULEPK028 — (RULE  1 48  CULTURE-1 ) 

RULE  148  CULTURE- 1: 

R?  ->  (BACTERIAL  868  ((RULE  148  CULTURE-1)  (RULE501  CULTURE- 1 )  (RULE500 
CULTURE-1)  (RULE602  CULTURE-1))  NIL) 


[Bacterial  is  concluded  by  ruleJ48,  but  other  d-rules  make  the  same  conclusion,  so 
this  is  only  weak  evidence  (400)  that  the  student  used  rule!48  to  form  his 
hypothesis....} 

[Applying  tutorial  rule  28.06] 

[RULEUSEOCF  of  (RULE1 48  CULTURE- 1)  »  400] 
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{After  all  of  the  d-rules  are  considered  separately,  "self -referencing'  strategies 
reconsider  the  evidence  for  the  model.  In  particular,  t-rule29.02  detects  that  the 
student  model  (USE-HISTORY)  disagrees  with  evidence  from  the  hypothesis. 
T his  is  recorded,  but  no  use  is  made  of  it  in  the  current  system .} 

RULEUSEDSREF.RULEPK029--(RULE  1 48  CULTURE-1 ) 

[Student  model:  RULE  1 48  CULTURE-1:  VALUE  =  0] 

[Applying  tutorial  rule  29.02  (else)] 

[Adding  RULE  148  CULTURE-1  to  MODELSURPRISE  agenda.] 

[USEDSREF  of  (RULE  148  CULTURE- 1)  *  400] 


{Evidence  from  the  hypothesis  (USED?)  modifies  the  student  model  (USE- 
HISTORY).) 

[SModel  update  for  RULE  1 48:  0  ->  400] 


{The  evidence  turns  out  to  be  decidedly  mixed,  so  the  tutor  asks  the  student  to 
support  his  hypothesis....) 

What  facts  about  this  case  tell  you  that  the  type  of  the  infection  is 
bacterial  (.9)? 

(Enter  one  factor  per  line.) 

*«  CSFGLUC 
**  PMNS 

The  percent  of  PMN's  in  the  CBC  is  not  relevant  for  determining  the  tvpe  of 
the  infection. 

»  CSFPOLY 
»«  PROTEIN 


{Alas,  the  student  has  indeed  left  out  the  evidence  contributed  by  rule  148,  based  on 
that  missing  information  about  partial  treatment.) 

You're  right  about  the  CSF  protein,  the  percent  of  PMNs  in  the  CSF  and  the 
CSF  glucose  value,  however  you  should  have  mentioned  the  fact  that  the 
meningitis  is  partially  treated  (.95),  the  WBC  from  the  CSF  and  the  fact 
that  the  simultaneous  blood  glucose  value  is  not  known. 
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To  summarize,  we  can  conclude  that  the  type  of  the  infection  is: 
bacterial  (considering: 

whether  the  meningitis  is  partially  treated  [RULE  148], 
the  WBC  from  the  CSF  and  the  percent  of  PMNs  in  the  CSF 
[RULE601], 


{GUIDON  has  told  the  student  about  rulel48  now,  so  it  assumes  that  he  has  now 
" applied  the  rule.'} 

[Assuming  that  S  can  now  apply  RULE  148  CULTURE- 1] 


{Later,  the  student  asks  for  a  summary  of  evidence  about  the  type  of  the  infection. 
One  rule  mentioned  is  ruleI48  which  GUIDON  thinks  the  student  has  considered.} 

Summary  of  evidence  that  the  type  of  the  Infection  Is  bacterial  (.98)  viral 
(-.76)  fungal  (-.83)  tb  (-.83): 

29a.  The  fact  that  the  meningitis  is  partially  treated  (.95)  is  evidence 
that  the  type  of  the  infection  is  bacterial  (.19)  [RULE  148] 

29b.  The  WBC  from  the  CSF  and  the  percent  of  PMNs  In  the  CSF  Is  evidence 
that  the  type  of  the  infection  is  bacterial  (.9)  viral  (-.6)  fungal 


{It  turns  out  that  the  student  was  so  excited  about  his  correct  evidence 
{CSF POLY),  he  didn’t  read  the  tutor’s  remarks  about  the  evidence  he  missed 
(rulel48)f  When  the  student  asks  for  details  about  rule!48,  the  program  notes 
that  two  goals  have  not  been  discussed,  and  offers  to  review  evidence  for  them.  As 
we  might  expect,  the  student  wants  to  know  about  the  evidence  for  "partial 
treatment ."} 

**  DETAILS  29A 

For  which  of  the  following  factors  would  you  like  to  see  a  review  of  the 

evidence? 

(select  from  the  following): 


% 
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1 )  the  infection  which  requires  therapy 

2)  the  fact  that  the  meningitis  is  partially  treated  (.95) 


Summary  of  evidence  that  the  meningitis  Is  partially  treated  (.95): 

32a.  The  infection  which  requires  therapy,  the  fact  that  organisms  were  not 
seen  on  the  stain  of  the  pending  csf  culture  (CULTURE-1)  and  the  time 
since  therapy  with  the  cephalothin  (DRUG-1)  was  started  is  evidence 
that  the  meningitis  Is  partially  treated  (.95)  [RULE  145] 


{ Now  that  rulel4i  has  been  shoum  to  the  student,  we  check  to  see  whether 
information  about  ' partial  treatment'  changes  our  model  of  what  he  knows.  In 
fact,  it  does  not:  rulel48  was  presented  during  hypothesis  evaluation  and  was 
marked  as  applied  at  that  time.) 

[Assuming  that  S  can  now  apply  RULE145  CULTURE-1] 

[Reconsidering  S's  use  of  PTMEN  CULTURE- 1] 

Do  you  want  to  see  RULE  148? 

«*  NO 


8.3. 2.2  Implementation 

A  d-rule,  whether  It  succeeds  or  falls,  is  said  to  fire  when  MYCIN  has  completed 
all  of  the  subgoals  required  by  the  left-hand  side  of  the  rule.  A  goal  is  complete  when 
ail  of  the  rules  that  might  update  it  have  fired,  and  case  data  has  been  acquired,  if 
necessary.  The  AND/OR  solution  tree  is  structured  by  inverse  pointers  that  permit  the 
program  to  repetitively  reconsider  d-rules  (indexing  them  by  the  case  data 
referenced  in  the  premise  part),  without  the  high  cost  of  reinterpreting  premises  from 
scratch.  Associated  with  each  goal  pursued  by  MYCIN,  and  hence  each  potential  topic 

in  the  tutorial  dialogue,  is  a  dynamic  inference  end  discussion  record.1 

1  Internally  it  is  represented  as  s  doubly-indexed  property  list  E.g,  the  topic 
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When  a  goal  is  completed,  GUIDON  marks  it  (EXPERTDONE  property),  and  checks 

to  see  if  each  of  the  d-rules  that  need  it  can  now  fire  (the  USE-SUCC!  and  CAUSE- 

FAIL!  properties  of  the  goal).  Each  d-ruie  has  a  logical  description  of  the  goals  that 

must  be  done  before  it  can  fire  (SUCC-PARMS!  or  FAIL-PARMS!,  depending  on  whether 

the  rule  succeeds  or  fails).  For  example,  the  (SUCC-PARMS!)  description  for  rule578 

(see  Figure  5.2)  applied  in  the  context  of  culture- 1  is: 

(AND  (TREATINF  CULTURE -1 ) 

(OR  (SPECSTAIN  CULTURE-1) 

(EXAMSTAIN  CULTURE-1)) 

(TYPE  CULTURE- 1) 

(BURNED  PATIENT-536)) 

This  rule  will  fire  as  soon  as  TREATINF,  TYPE,  BURNED  and  either  SPECSTAIN  or 
EXAMSTAIN  are  complete. 


When  a  successful  rule  fires,  several  operations  take  place: 

1 )  Mark  tha  rufa  (FIRED  property) 

2)  Check  tha  student  profile  to  see  If  he  wants  to  be 
Informed  of  the  rule's  application. 


3)  Invoke  SAPPLIED.RULEPK04 1  to  decide  whether  the 
student  applied  the  rule  (described  in  Section  8-3.2.3 
below). 


4)  For  each  goal  that  the  rule  concludes  about,  modify  the 
expert’s  current  hypothesis  about  this  goal  by  incorporating  the 
new  evidence.  If  GUIDON  strongly  believes  that  the  student  applied 
the  rule,  then  modify  the  student  inference  record  in  the 

same  way.' 


"identity  of  organism"  has  a  record  for  each  relevant  context:  organism- 1  (perhaps 
from  a  blood  culture),  organism-2  (perhaps  from  the  sputum),  and  so  on. 

1  Thus,  four  values  are  associated  with  each  goal  of  the  solution  tree:  MYCIN's 
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5)  Hint  about  or  directly  state  the  new  evidence,  according  to 
the  student  profile. 


6)  For  each  goal  that  has  been  updated  (perhaps  by  more  than 
one  conclusion),  check  fo  see  It  the  goeJ  It  now  complete. 


The  procedure  for  checking  to  see  if  a  goal  is  complete  after  a  rule  that  updates 
it  succeeds  is  as  follows. 


1)  Mark  the  list  of  rules  that  update  this  goal  (UPDATE* 
SUCC!)  by  moving  the  rule  that  just  fired  past  the  DONE  MARK.' 


2)  If  self*referencing  rules  can  now  be  applied  (SREFMARK  at 
the  head  of  both  UPDATE-SUCC!  and  UPDATE-FAIL!,  and  the  student 
has  requested  the  value  of  the  parameter  if  it  is  LABDATA),  then 
allow  rules  that  have  been  waiting  to  fire  to  ripple  their 
conclusions.  (Self-reference  rules  are  caught  just  before  they 
fire,  and  saved  on  the  SREF-WAITING  property  of  the  goal.) 


3)  If  the  goal  Is  now  complete,  then  recur  (check  the  rules 
that  use  It  to  see  if  they  now  fire).  A  goal  is  complete  when  its 
rules  are  done,  and  the  student  has  already  requested  the  value,  if 
it  is  case  data  that  needs  to  be  asked  for  this  case.  If  the  rules 
did  not  contribute  a  final  value,  so  MYCIN  would  now  ask  for  a  value, 
the  goal  is  marked  (NEED-TO-ASK  property).  Thus,  t-rules  that  say 
"it  Is  appropriate  to  request  the  reported  value  of  the  goal"  will 
succeed  If  the  goal  is  LABDATA  that  the  MYCIN  asked,  but  the 
student  has  not  yet  asked,  or  the  goal  Is  marked  as  NEED-TO-ASK 
(rules  done,  must  ask  now).  Figure  B.3  shows  protocol  excerpts 
that  illustrate  this  underlying  process. 


current  hypothesis,  the  student's  current  hypothesis  (this  is  GUIDON'S  belief  and  not 
to  be  confused  with  explicit  hypotheses  stated  by  the  student),  the 
(cumulative)  evidence  that  has  been  discussed  to  this  point  in  the 
dialogue,  and  the  final  value  that  MYCIN  will  eventually  believe. 

1  UPDATE-SUCC!  Is  of  the  form:  <rules  ...  SREFMARK  ...  self-reference  rules  ... 
DONEMARK  ...  fired  rules>  Thus,  self-referencing  rules  cannot  fire  until  SREFMARK  is  at 
the  head  of  the  list.  The  rules  for  the  goal  are  done  when  SREFMARK  and  DONEMARK 
are  together  at  the  head  of  the  Hat. 
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The  procedure  for  updating  the  model  when  a  failing  rule  fires  is  much  simpler. 
No  conclusions  are  rippled,  and  the  student  model  is  not  affected.  However,  we  still 
check  to  see  if  the  goal  is  complete,  or  self  referencing  rules  can  be  applied. 

**  ALCOHOLIC 

GIVENPARMDATA. PROC012--( ALCOHOLIC  PATIENT-538)  NIL  T 
[Applying  tutorial  rule  12.01  (else)] 

[Applying  tutorial  rule  12.03] 

NONEEDTOASK . RULE  PK032-- ( ALCOHOLIC  PATIENT- 538 ) 

[Applying  tutorial  rule  32.05] 

PREMATURE-QUESTION : 

While  you  will  eventually  find  it  useful  to  ask  about  whether  Pt538  is  an 
alcoholic,  it  is  premature  to  do  so  now.  There  is  still  a  possibility  that 
other  data  will  enable  you  to  deduce  a  value. 


**  USE  42C 

Pt538  is  a  compromised  host. 

[Adding  ALCOHOLIC  PATIENT-538  to  NEEO-TO-A5K  agenda.] 

[Adding  IMMUNOSUPPRESSED  PATIENT-538  to  NEED-TO-ASK  agenda.] 

Figure  8.3.  Protocol  Excerpts  Showing  Effect  of  the  Overlay  Model 
on  Date  Acquisition 


8. 3.2.3  Deciding  Whether  the  Student  Applied  a  D-Rule 

Special  t-rules  (SAPPLIED.RULEPK041 )  for  updating  the  overlay  model  are 
invoked  whenever  the  expert  program  successfully  applies  a  d-rule.  These  t-rules 
determine  the  tutor's  belief  that  the  student  has  applied  the  d-rule.  This  decision  is 
based  upon: 

1 )  The  record  of  previous  interactions  with  the  student  (USE- 
HISTORY)  or  the  a  priori  (background)  evidence. 
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2)  If  the  rule  was  discussed  during  this  session,  then  this  is 
weak  evidence  that  the  student  has  taken  the  rule  into 
consideration. 


3)  if  the  tutor  believes  that  the  student  doesn't  know  how  to 
achieve  the  subgoals  that  appear  In  the  d-rule  (factors  that 
'  require  the  application  of  rules),  the  program  concludes  with 
certainty  that  the  student  has  not  applied  the  rule. 

To  determine  whether  the  student  can  achieve  subgoals  that  appear  in  the  d- 
rule  (criterion  3),  each  clause  of  the  rule  is  evaluated  (by  interpreting  the  source 
code),  using  the  model  of  the  student's  knowledge,  rather  than  the  conclusions  used 

by  MYCIN.1  If  the  student  cannot  apply  a  clause,  then  this  information  is  saved  for 
later  use  in  tutoring  (SCANT APPLY  property  of  the  rule). 

It  is  important  that  the  tutor  revise  the  student  model  as  the  dialogue  proceeds. 
In  particular,  if  the  program  later  gives  information  to  the  student  that  enables  him  to 
apply  a  rule,  then  the  effect  of  this  rule  must  be  taken  into  account.  For  example,  the 
protocol  excerpt  in  Section  8.3.2.1  illustrated  that  the  tutor  considered  the  student's 
use  of  evidence  for  "partial  treatment”  after  it  told  him  about  rule145.  In  practice, 
reconsideration  occurs  when  a  rule  is  marked  as  "discussed"  (presented  in  a  summary 
of  evidence,  quizzed  about,  worked  through  clause  by  clause  in  the  dialogue,  and  so 
on). 


The  reconsideration  process  is  undertaken  when  the  model  Indicates  that  the 
rule  was  not  applied  by  the  student  (SAPPIHED?  less  than  STRONGCF).  Each 


1  Each  rule  predicate  eventually  must  invoke  the  MYCIN  function  that  accesses 
dynamic  patient  data.  A  global  flag  is  set  that  directs  this  function  to  use  the  overlay 
data  base  (evidence  that  the  tutor  believes  Is  hypothesized  by  the  student)  Instead 
of  MYCIN's  context  tree. 
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conclusion  made  by  the  rule  is  allowed  to  modify  the  record  of  the  student's 
Inferences  (ASSUMEOSVAL  property  of  the  goal).  (The  tutor  prints  "[Assuming  that  S 
can  now  apply  <rule>.”)  Then,  If  the  expert  is  done,  the  tutor  reconsiders  the  rules 
that  use  these  modified  goals.  That  is,  each  rule  (on  the  USE*SUCC!  list  of  the  goal) 
is  reevaluated  by  the  rule  packet  that  determines  if  the  student  can  apply  the  rule 
(SAPPLIED.RULEPK041).  If  the  rule  now  fires  (using  the  student  inference  record), 
the  process  recurs,  with  reconsideration  of  all  rules  that  use  goals  that  have  been 
modified  and  have  been  completed  by  the  expert. 

Jn  many  of  the  tutorial  dialogues  we  have  seen  so  far,  this  reconsideration 
process  is  not  very  important.  As  shown  in  the  typescript,  the  higher  rules  (like 
rule148)  are  usually  discussed  explicitly  before  the  student  requests  details,  and  so 
finds  out  about  the  lower  rules  (like  rule  145).  When  the  program  reconsiders  the 
student's  use  of  lower  goals,  it  finds  that  the  higher  rules  are  marked  as  discussed, 
and  so  were  probably  already  "applied." 

8. 3. 2.4  Complications:  Interaction  between  Modeling  and  the  Dialogue 

It  would  be  convenient  If  GUIDON  could  insist  that  the  dialogue  follow  the 
AND/OR  tree  exactly.  This  would  mean  that  a  goal  would  not  be  marked  as  discussed 
until  all  of  its  rules  had  been  discussed,  and  a  rule  would  not  be  marked  as  discussed 
until  all  of  Its  subgoals  had  been  discussed.  However,  in  the  interests  of  time  and 
economical  presentation,  GUIDON  doesn't  always  wait  for  MYCIN  to  apply  a  rule  or  to 
finish  a  goal  before  telling  the  student  about  it  and  moving  on  to  another  topic. 
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First,  it  turns  out  that  it  is  sometimes  convenient  to  present  a  rule's  conclusion 
without  formally  discussing  the  case  data  mentioned  in  the  premise.  This  is  illustrated 
below  (compare  to  page  76). 


*«  HELP 

Try  to  determine  whether  Pt638  has  a  head  injury  defect. 

GOAL.PROCOOI  --(HEADINJDEF  PATIENT-638)  (COVERFOR  CULTURE-1)  T 
[Applying  tutorial  rule  1 .02] 

DEFNRULE.RULEPK042— (HEADINJDEF  PATIENT-638)  (RULE609  PATIENT-638) 


[GUIDON  decides  to  discuss  the  single  definitional  rule  instead  of  opening  the 
goal  for  discussion.  The  student  model  indicates  that  the  student  knows  this 
rule....} 

[Applying  tutorial  rule  40.01] 

[Applying  tutorial  rule  40.02] 

[Backgroundcf  of  RULE509  >  940] 

[SModel  update  for  RULE609:  NIL  ->  940] 

[Student  model:  RULE609  PATIENT-638:  STRONG  =  T] 


[Rather  than  discussing  the  rule  step-by-step,  the  tutor  presents  it  directly....} 
[Applying  tutorial  rule  42.01] 

PRESENTVALS.RULEPK026-NIL  (HEADINJDEF  PATIENT-638)  (RULE609  PATIENT-63B) 
[Decided  that  RULE509  PATIENT-638  is  on  the  lesson  plan.] 

[Student  model:  RULE609  PATIENT-638:  STRONG  *  T] 


[The  student  probably  knows  the  rule,  so  it  is  stated,  rather  than  posed  as  a 
question....} 

[Applying  tutorial  rule  26.02] 

The  fact  that  Pt638  has  not  had  an  injury  or  Insult  to,  or  defect  In  the  CNS 
is  evidence  that  Pt638  does  not  have  a  head  injury  defect  [RULE609]. 


Growth  and  Revision  of  the  Model 


297 


{ However ,  the  tutor  must  now  force  MYCIN  to  fire  this  rule,  since  the  normal 
procedure  for  giving  case  data  (injury  to  CNS)  was  not  followed.) 

[Forced  discussed  rule  RULE509  PATIENT-638  to  fire.] 

[Applying  tutorial  rule  41.01  (else)] 

[Applying  tutorial  rule  41.02] 

[Student  applied  (RULE609  PATIENT-638)  «  976  ] 

[S  applied  RULE609  PATIENT-638] 


Of  course,  the  record  of  case  data  given  to  the  student  must  still  be  maintained 
when  rules  are  presented  in  this  way.  Similar  problems  occur  when  rules  that  have 
not  fired  are  posed  as  questions  (see  Appendix  B). 

Second,  a  goal  might  be  marked  as  discussed  before  all  of  is  rules  have  fired. 
This  occurs  when  the  topic  is  finished  prematurely  ("substantially  done"  heuristic)  or 
when  the  student  requests  for  discussion  to  end  (TELLME  or  IKNOW  option).  It  was 
necessary  to  tell  MYCIN  to  use  evidence  about  the  type  of  the  Infection  as  if  it  had 
already  received  Its  final  value,  just  after  the  student  used  the  TELLME  option  (page 
61). 


Finally,  it  is  possible  for  MYCIN  rules  to  conclude  about  some  goal  that  has 
already  been  marked  as  completed.  This  redundancy  is  caused  by  rules  that 
deliberately  change  the  value  of  parameter  (during  a  consultation)  to  have  some  side 
effect.  Fortunately,  there  is  only  one  rule  of  this  type  In  MYCIN.  Nevertheless, 
GUIDON  must  detect  this  situation  to  keep  the  overlay  model  consistent.  This  excerpt 
shows  the  error: 


»>  61a.  MYCIN  has  made  a  FINAL  CONCLUSION  about  the  number  of  cultures 
from  this  site  since  the  onset  of  the  infection. 
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Possible  error  in  rules:  RULE049  CULTURE-1  updates  a  done  parm:  TREATINF 
CULTURE- 1 

Possible  error  in  rules:  RULE049  CULTURE- 1  updates  a  done  parm:  WHENSYM 

CULTURE- 1 

RULE-RETURN: 

(Continuing  our  discussion  of  whether  the  organisms  isolated  from  the 
pending  csf  culture  (CULTURE-1)  should  be  considered  for  therapy  [RULE049].) 


It  should  be  noted  that  waiting  until  a  goal  is  completed  by  MYCIN  before 
updating  the  model  may  be  unrealistic  in  that  the  student  may  be  proceeding  on  partial 
evidence,  and  even  controlling  his  search  by  pushing  tentative  conclusions  forward. 
For  GUIDON  to  do  the  same,  the  modeling  process  would  require  rippling  intermediate 
conclusions  into  the  rules  that  use  them,  rather  than  waiting  until  a  final  conclusion  for 
the  goal  had  been  reached.  For  example,  as  soon  as  there  is  evidence  for  a  fungal 
Infection,  all  rules  that  use  this  information  would  be  examined  to  see  if  new 
conclusions  can  be  drawn.  There  is  no  theoretical  problem  that  prevents  us  from  using 
MYCIN's  rules  for  modeling  in  this  way,  but  it  would  be  a  very  slow  process.  It  may  be 
better  to  use  this  refinement  of  the  modeling  technique  when  less  expensive 
assumptions  fail  to  account  for  the  student's  behavior  [Burton,  1979]. 

8.3.2.Q  Credit  Assignment!  Response  to  a  Student  Hypothesis 

When  GUIDON  attempts  to  account  for  student  behavior  in  terms  of  MYCIN's 
rules  It  finds  that  the  evidence  Is  usually  ambiguous.  Even  if  we  restrict  the  behavior 
we  are  modeling  to  that  of  making  an  hypothesis,  as  we  have  in  GUIDON,  the  available 
evidence  does  not  necessarily  implicate  a  unique  set  of  rules.  A  d-rule  might  make 
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several  conclusions  and  the  student  might  mention  only  some  of  them.  A  conclusion 
might  be  made  by  more  than  one  rule.  Therefore,  the  problem  is  to  take  the  student 
hypothesis,  the  rules  applied  by  MYCIN  and  their  conclusions,  and  to  construct  an 
interpretation  of  which  rules  best  account  for  the  hypothesis.  This  interpretation  is 
constructed  by  special  t-rules.  In  this  section,  we  describe  these  t-rules  and 
consider  related  issues  such  as  the  special  problem  of  modeling  behavior  that  is  itself 
based  on  uncertain  judgments,  the  interaction  of  causal  and  cookbook  domain 
knowledge,  and  the  problem  of  student  forgetting. 

For  the  purpose  of  illustration,  we  will  consider  the  following  student  hypothesis 

about  the  organisms  causing  the  infection:1 

"I  think  that  the  organisms  present  are  DIPLOCOCCUS-PNEUMONIAE, 
PSEUDOMONAS- AERUGINOSA,  and  NEISSERIA-MENINGITIDIS.  ■ 

Figure  8.4  illustrates  how  the  overlay  model  is  updated  for  this  hypothesis.  As 
mentioned  above,  more  than  one  interpretation  Is  possible.  For  example,  three  rules 
conclude  Pseudomonas,  rules  645,  657,  and  678;  any  combination  of  these  or  none 
might  have  been  considered  by  the  student. 


1  GUIDON  was  interrupted  during  a  tutorial,  and  its  record  of  the  session  was 
modified  to  simulate  specific  modeling  problems.  Thus,  the  situation  was  contrived,  but 
t-rules  were  actually  applied  in  the  way  we  describe  here. 
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GOAL:  COVERFOR 


Figure  8.4.  Constructing  the  Overlay  Model  from  a  Student  Hypothesis 


Key:  O-rules  that  conclude  about  organisms  to  "cover  for"  are  shown  with  their 
key  factors.  Circled  values  are  missing  from  the  student's  hypothesis  (e.g.  E.coli)  or 
wrongly  stated  (e.g.  Neisseria).  Dotted  lines  lead  from  rules  the  student  probably  did 
not  use.  iH  is  an  evidence  link  that  the  tutor  decided  is  unknown  to  the  student;  R  and 
W ,  links  to  right  and  wrong  values  that  the  tutor  believes  are  known  by  the  student;  /, 
unique  link  (expert  knows  of  no  other  evidence  at  this  time);  t,  questionable  (tutor  is 
not  certain  which  evidence  was  considered  by  the  student).  Thus,  Rf  means  that  the 
student  stated  this  value,  It  Is  correct,  and  more  than  one  d-rule  supports  it. 
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A  packet  of  T-rules  (RULEUSED.RULEPK028)  determines  how  strongly  the  tutor 
believes  that  the  student  has  taken  each  d-rule  that  MYCIN  applied  into  account.  The 
t-rules  that  succeed  will  modify  the  cumulative  belief  that  the  given  d-rule  was 
considered  by  the  student  (the  USED?  property  of  the  d-rule).  In  this  example,  we 
see  that  T-rule  28.06  succeeded  when  applied  to  d-rules  645  and  657.  While  the 
student  mentioned  a  value  that  they  conclude  (pseudomonas)  (clause  1  of  the  t-rule), 
he  missed  others  (clause  3).  Moreover,  he  did  not  mention  values  that  can  ONLY  be 
concluded  by  these  d-rules  (clause  2),  so  the  overall  evidence  that  these  d-rules 

were  considered  is  weak  (-.70). 1 

T-RULE28 .85 

If:  1)  The  hypothesis  does  Include  values  that  can  be  concluded  by 
this  d-rule,  as  well  as  other  rules,  and 

2)  The  hypothesis  does  not  Include  values  that  can  only  be 
concluded  by  this  d-rule,  and 

3)  There  are  values  concluded  by  the  d-rule  that  are  missing  from 
the  hypothesis 

Then:  Define  the  belief  that  this  rule  was  considered  to  be  -.7 

After  each  of  the  d-rules  applied  by  MYCIN  is  considered  independently,  a 
second  pass  Is  made  to  look  for  patterns.  Two  t-rules  from  this  second  rule  packet 
(RULEUSEDSREF.RULEPK029)  are  shown  below.  T-rule  29.01  applied  to  d-rule  678:  of 
the  d-rules  that  conclude  pseudomonas,  this  is  the  only  one  that  is  believed  to  have 
been  considered,  thus. increasing  our  belief  that  d-rule  678  was  used  by  the  student. 
T-rule  29.06  applies  to  d-rules  646  and  661:  the  factor  NOSOCOMIAL  appears  only  in 
their  premises,  and  they  are  not  believed  to  have  been  considered.  This  is  evidence 


1  As  usual,  the  certainty  factor  of  -.70  was  chosen  by  the  author.  Experience 
with  MYCIN  shows  that  the  precise  value  is  not  important,  but  the  scale  from  -1  to  1 
should  be  used  consistently. 
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that  NOSOCOMIAL  was  not  considered  by  the  student,  increasing  our  belief  that  each 
of  the  d-rules  that  mention  it  were  not  considered. 

T-RULE29.81 


If:  You  believe  that  this  rule  was  considered,  it  concludes  a  value 
present  in  the  student’s  hypothesis,  and  no  other  rule  that 
mentions  this  value  is  believed  to  have  been  considered 
Then:  Modify  the  cumulative  belief  that  this  rule  was  considered  by 
.4 


T-RULE29 .85 


If:  This  domain  rule  contains  a  factor  that  appears  in  several  rules, 
none  of  which  are  believed  to  have  been  considered  to  make  the 
hypothesis 

Then:  Substep  1.  Modify  the  cumulative  belief  that  this  rule  was 
considered  by  -.3 

Substep  11.  Record  on  the  agenda  that  this  domain  rule  contains 
a  factor  that  appears  in  several  rules,  none  of 
which  are  believed  to  have  been  considered  to  make 
the  hypothesis. 


GUIDON  records  that  NOSOCOMIAL  Is  a  "missed  subgoal,"  but  no  t-rules  have 
been  developed  for  using  this  information  in  tutoring.  We  want  to  collect  more  data 
about  the  incidence  and  correctness  of  this  part  of  the  model  before  Incorporating  it 
in  specialized  rules  for  guiding  the  dialogue.  In  fact,  experience  with  the  current  t- 
rules  is  too  limited  to  report  on  their  validity  at  this  time,  though  It  seems  plausible 
that  they  form  a  basic  set  of  considerations  that  should  be  incorporated  in  the  model 
in  one  form  or  another. 

It  is  possible  to  speculate  on  refinements  to  the  current  set  of  modeling  rules. 
For  example,  the  excerpt  from  the  protocol  (Section  8.3.2. 1)  Illustrates  that  the 
history  of  the  student's  use  of  options  with  respect  to  a  particular  d-rule  might  be 
useful  Information  to  include  in  the  model.  If  the  student  asks  for  details  about 
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evidence  for  a  conclusion,  then  we  know  that  he  has  at  least  read  the  indexed 
remark.  On  the  other  hand,  If  these  details  were  provided  earlier,  then  we  can  deduce 
he  forgot  them,  did  not  understand  them,  or  may  not  have  read  a  prolonged  tutorial 
statement.  Indeed,  the  excerpt  demonstrates  that  forgetting  and  not  reading  remarks 
could  defeat  even  very  sophisticated  modeling  rules.  Review,  follow-up  quizzes,  and 
checking  for  consistency  (by  option  use)  are  aspects  of  modeling  that  could  be 
usefully  incorporated  in  tutoring. 

Besides  using  other  sources  of  evidence  In  the  model,  the  model  might  be  based 
on  d-rules  that  fail,  or  even  simple  variations  on  MYCIN's  rules,  should  it  be  impossible 
to  adequately  explain  an  hypothesis  in  terms  of  the  successful  rules  alone.  For 
example,  perhaps  the  student  is  not  considering  an  optional  clause  (as  NOSOCOMIAL 
might  be  considered  to  be  optional  in  rule561— see  Section  S.2.4.6).  Similar  errors 
include  applying  a  rule  in  the  wrong  context  (using  meningitis  Information  in  a  cystitis 
case),  forgetting  case  data,  or  classifying  evidence  incorrectly  (tabular  rule). 

Of  course,  It  Is  not  quite  right  to  think  of  student  errors  solely  in  terms  of  the 
syntax  of  a  rule.  As  the  examples  above  illustrate,  we  don't  believe  that  a  student's 
knowledge  is  randomly  different  from  expert  rules.  This  is  possible,  but  it  seems  more 
plausible  that  there  is  a  reason  that  the  student  has  disregarded  or  varied  a  condition. 
The  model  of  what  constitutes  satisfying  evidence  provides  one  basis  for  generating 
plausible  student  errors.  For  example,  the  student  might  be  disregarding  the  context 
(meningitis  versus  cystitis),  omitting  a  condition  that  tightens  evidence  (NOSOCOMIAL), 
or  confused  about  the  principle  of  the  Inference  (wrong  belief,  wrong  conclusion  in  a 
tabular  rule). 
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Taking  our  analysis  of  student  errors  a  step  further,  accurate  modeling  might 
include  interactions  among  the  rules.  Common  errors  that  we  might  expect  include 
over-generalizations  and  false  analogies.  We  should  be  aware  that  attempting  to 
explain  the  student's  hypotheses  solely  in  terms  of  MYCIN's  performance  rules  might 
result  in  missing  the  reasoning  steps  that  account  for  the  student's  behavior.  In 
particular,  the  student  may  be  forming  an  association  that,  on  the  surface,  we  do  not 
recognize.  Yet,  this  association  could  be  based  on  the  causal  arguments  that  support 
MYCIN's  rules.  Following  the  terminology  of  [Brown,  Collins,  &  Harris,  1977]  (AILS), 
we  cannot  construct  a  "deep  model"  of  the  student's  reasoning  by  using  MYCIN's 
diagnostic  rules  alone.  It  may  be  necessary  to  consider  causal  and  mechanistic 
arguments  in  order  to  explain  an  hypothesis.  An  example  is  presented  in  Section 
8.6. 


8.3.3  Revision  of  Belief 

Recall  that  there  are  three  components  In  GUIDON'S  student  model:  the 
cumulative  belief  that  the  student  knows  the  rule  (USE-HISTORY),  the  belief  that  he 
applied  the  rule  In  a  given  case  (SAPPLIED?),  and  the  belief  that  the  rule  was  used  to 
generate  an  hypothesis  for  a  goal  (USED?).  We  have  considered  background 
evidence  for  initializing  USE-HISTORY,  and  modeling  rules  for  determining  and 
maintaining  SAPPLIED?  and  USED?.  It  remains  for  us  to  consider  how  USE-HISTORY, 
the  cumulative  model,  is  revised.  The  evidence  for  the  model  includes  the  student's 
use  of  options  (particularly,  in  this  version  of  GUIDON,  his  request  for  assistance),  his 
response  to  direct  questions  about  the  d-rules,  and  his  hypotheses  for  subproblems. 
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As  illustrated  in  the  overview  diagram  (Figure  8.2),  the  revision  process  has  the 
nature  of  a  feedback  operation:  the  model  of  what  the  student  knows  (and/or  is 
doing)  drives  t-ruies  that  press  the  student  to  reveal  more  of  what  he  knows.  The 
revision  problem  is  to  incorporate  this  new  evidence  into  the  model. 

Programs  like  WEST  and  WUMPUS  maintain  a  simple  ratio  of  the  number  of  times 
the  student  showed  evidence  of  using  the  rule  (skill  or  issue)  divided  by  the  number 
of  times  it  was  appropriate  to  use  the  rule.  In  keeping  with  our  experimental  use  of 
certainty  factors  for  representing  belief,  GUIDON  modifies  the  model  by  combining 
USE-HISTORY  with  USED?  after  an  hypothesis.  Similarly,  belief  that  the  student  knows 
a  rule  is  decreased  (by  -.4)  when  GUIDON  chooses  to  present  the  rule  in  response  to 
a  student  request  for  assistance.1  While  the  CF  combining  function  will  give  a  value 
different  from  taking  a  USED/APPROPRIATE  ratio,  the  change  in  belief  Is  in  the  same 
direction.  It  Is  doubtful  that  the  precise  value  is  of  much  significance,  though  the 
accumulation  of  belief  Is  probably  important,  since  It  seems  intuitively  plausible  that 
teachers  consider  how  frequently  a  student  performs  appropriately. 

Do  certainty  factors  allow  more  evidence  to  be  Incorporated  in  the  model  than 
the  USED/APPROPRIATE  ratio?  First,  observe  that  the  USED?  CF  could  easily  be 
converted  to  the  ratio  model  (if  USED?  is  above  the  "strong  evidence"  threshold, 
increment  USED  by  1,  otherwise  0).  Similarly,  a  wrong  answer  to  a  question  or  missed 
opportunity  to  apply  a  rule  (at  the  time  help  is  requested)  could  be  modeled  by 


1  GUIDON  chooses  the  rule  according  to  its  strategy  of  hypothesis  revision.  The 
student's  request  for  help  Is  taken  to  mean  that  he  doesn't  know  what  rule  to 
consider  next,  hence  he  doesn't  know  this  rule.  Of  course,  he  might  know  the  rule  and 
just  doesn't  want  to  consider  it  because  he  Is  following  a  different  strategy.  Or 
maybe  he  asked  for  help  Just  to  see  what  the  program  would  suggest. 
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incrementing  APPROPRIATE,  but  not  USED.  However,  the  USED/APPROPRIATE  ratio 
changes  the  model  in  the  same  way  each  time.  Certainty  factors  can  be  used  more 
flexibly,  allowing  finer  distinctions  to  be  made.  For  example,  the  tutor  might  want  to 
modify  its  belief  more  when  a  direct  question  is  missed  than  when  an  hvp<>u» 
seems  to  Indicate  that  a  rule  wasn't  used.  Indeed,  perhaps  the  model  sIkm.m  «.*• 
segmented  according  to  the  source  of  the  evidence,  and  reasoning  about  the 
student's  knowledge  should  be  done  more  symbolically,  based  on  the  source. 
However,  we  don't  know  at  this  time  what  kinds  of  distinctions  might  be  worth 
retaining,  or  how  they  could  be  used. 

There  is  actually  another  basic  problem  involved  in  revising  the  student  model. 
Revisions  based  on  the  student's  use  of  knowledge,  as  abstracted  from  his  behavior, 
pose  no  direct  problem  because  the  model  we  have  described  is,  by  design,  a 
predictive  indication  of  the  student's  use  of  knowledge.  We  at  least  know  which  way 
to  modify  the  model  when  new  evidence  comes  in:  If  his  behavior  Is  explained  by  the 
rule,  then  we  increase  our  belief  that  he  uses  the  rule  appropriately,  and  vice  versa. 
However,  a  quiz  does  not  measure  the  student's  use  of  knowledge,  so  much  as  his 
ability  to  recall  a  fact  or  relation  upon  demand.  The  distinction  is  particularly  clear  if 
you  consider  what  happens  when  a  student  answers  a  question  incorrectly  and  the 
right  rule  Is  presented  to  him.  Should  this  increase  or  decrease  the  tutor's  belief  that 
the  student  will  use  the  rule  appropriately  In  the  future? 

If  the  student  answers  a  question  correctly,  GUIDON  increases  USE-HISTORY  by 
.7  for  fill-in  and  multiple-choice  questions,  and  .2  for  true/false  questions.  If  the 
student  is  wrong,  USE-HISTORY  is  decreased  by  .2.  If  our  model  of  problem-solving 
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insisted  on  a  clear  distinction  between  retail  and  use,  we  would  have  to  keep  a 
separate  tally  lor  quizzes.  So  we  might  say,  "here  is  GUIDON'S  belief  that  the  student 
will  answer  a  question  about  this  rule  correctly,"  and  "here  Is  GUIDON'S  belief  that  the 
student  will  consider  this  rule  (on  his  own)  when  presented  with  a  case  in  which  the 
rule  is  appropriate.”  However,  the  ability  to  recall  a  rule  seems  to  be  evidence, 
however  weak,  that  the  rule  will  be  used  appropriately  as  well.  This  is  why  we  feel 
justified  in  combining  the  measures  in  this  first  version  of  the  program. 

Regardless  of  whether  there  are  two  measures  or  one,  it  Is  not  clear  how  a 
corrected  wrong  answer  will  affect  future  student  performance.  One  could  argue  that 
once  the  student  is  corrected,  he  will  learn  the  rule  and  use  it  correctly  next  time. 
However,  GUIDON  models  conservatively  by  assuming  ignorance.  Revising  the  belief 
model  intelligently  might  require  consideration  of  individual  differences  in  learning  and 
comprehension,  a  model  of  forgetting,  and  certainly  some  measure  of  how  hard  it  is  to 
incorporate  the  particular  rule  in  behavior  (on  long  and  short  term  bases). 

8.4  Uses  of  the  Model 

Many  of  the  t-rules  presented  in  Chapter  6  make  use  of  the  student  model. 
These  uses  are  summarized  below. 

1)  Economical  discussion.  The  model  allows  the  tutor  to 
match  time  and  detail  of  a  discussion  to  the  needs  of  a  student. 

This  is  particularly  important  to  avoid  boring  the  student  by 
questions  and  protracted  consideration  of  material  that  he  already 
knows. 


2)  Responding  to  an  hypothesis.  Given  the  classification  of 
d-rules  into  used,  possibly  used,  and  missed,  the  tutor 
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congratulates  the  student  and  tests  him.  The  model  enables  the 
tutor  to  selectively  match  its  response  to  the  needs  of  the  student, 
exhibiting  both  economical  and  opportunistic  strategies. 


3)  Opportunistically  examining  understanding.  The 
feedback  nature  of  tutoring  is  illustrated  well  by  opportunistic 
tutoring  strategies.  One  reason  GUIDON  interrupts  the  student  to 
present  new  facts  or  quiz  him  is  to  improve  the  model  of  what  he 
knows.  For  example,  the  pregram  quizzes  about  a  rule  that  uniquely 
accounts  for  a  student's  request  for  data  (when  the  student  model 
indicated  that  the  student  did  not  know  the  rule).  From  this 
perspective,  one  goal  of  the  tutor  is  to  maintain  an  accurate  model 
of  what  the  student  knows  and  is  doing.  Thus,  the  model  drives 
certain  tutorial  actions  for  the  main  purpose  of  maintaining  its  own 
consistency.  Indeed,  the  perspective  that  tutoring  is  a  model¬ 
building  activity  suggested  GUIDON'S  current  set  of  opportunistic 
strategies  for  finding  and  resolving  conflicts  in  the  student  model 
(see  particularly  Section  6. 2.3.3). 


4)  Providing  assistance.  As  presented  in  the  introduction  to 
this  chapter,  it  is  useful  for  a  student  model  to  indicate  what  the 
student  is  doing  or  thinking  about,  as  well  as  what  he  knows  (in  a 
static  sense).  This  component  of  the  model  is  particularly  valuable 
when  providing  assistance  to  complete  a  subproblem.  We  would 
expect  a  responsive  tutor  to  pick  up  on  the  student's  partial 
solution,  rather  than  ignore  it  or  suggest  a  different  route  without 
comment.  (GUIDON'S  model  does  not  currently  incorporate  the 
student's  strategies.) 


It  should  be  noted  that  In  no  case  does  the  model  alone  make  the  tutor  efficient 
and  responsive.  On  the  other  hand,  a  battery  of  clever  instructional  methods  are  not 
sufficient  either  for  guiding  the  dialogue  intelligently.  Both  must  be  coupled  so  that 
tutoring  specifically  exercises  the  skills  that  need  work  (BIP  research  [Wescourt, 
Beard,  &  Gould,  1877]  [Wescourt,  et  al„  1977]  emphasized  this). 
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8.6  Inadequacy  of  Performance  Knowledge  for  Modeling  Misconceptions 

In  the  description  of  MYCIN's  knowledge  base  (Chapter  6),  we  emphasized  that 
the  rules  only  capture  a  superficial  layer  of  associations  between  evidence  and 
diagnosis.  We  gave  examples  of  mechanistic  reasoning  and  principles  that  implicitly 
justify  the  rules.  That  is,  arguments  for  the  validity  of  the  rules  refer  to  concepts  and 
relations  that  are  not  represented  in  the  knowledge  base.  Therefore,  it  seems 
plausible  that  when  a  student  (or  physician  for  that  matter)  is  asked  to  justify  an 
hypothesis,  he  may  offer  evidence  that  he  feels  is  in  turn  justified  by  underlying 
causal  relations  and  principles  that  he  has  learned.  When  the  student's  hypotheses 
and  support  are  wrong  because  his  underlying  reasoning  Is  faulty,  GUIDON  will  be  able 
to  detect  the  superficial  problem  (the  wrong  answer),  but  not  be  able  to  offer  any 
help  at  all  in  resolving  the  underlying  confusion.  In  this  respect,  GUIDON  tutoring 
capabilities  are  bounded  by  the  rule  set,  which  limits  modeling  to  a  relatively  shallow 
level  of  diagnostic  reasoning. 

Figure  8.6  is  an  excerpt  from  e  GUIDON  tutorial  that  Illustrates  the  limits  of  the 
program's  modeling  capability.  When  asked  to  support  his  (incorrect)  hypothesis  that 
the  patient  had  had  a  splenectomy,  the  student  offers  evidence  that  MYCIN  indicates 
is  Irrelevant.  Our  question  should  be,  why  did  the  student  think  that  whether  or  not 
the  patient  was  a  compromised  host  could  tell  him  whether  the  patient  had  had  a 
splenectomy?  We  will  examine  the  student's  reasoning  1  and  consider  what  additional 
knowledge,  if  any,  GUIDON  would  need  to  be  able  to  explain  to  the  student  where  he 
went  wrong. 

1  In  this  example,  the  author,  who  has  no  medical  training,  was  the  student.  The 
confusion  was  not  contrived. 
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ft*  KNOW  SPLENECTOMY 

Has  Pt538  had  a  splenectomy? 

•  *  NO 

What  facts  about  this  case  tell  you  that  Pt538  has  not  had  a  splenectomy? 
(Enter  one  factor  per  line.) 
ft»  COMPROMISED 

Whether  Pt538  Is  a  compromised  host  Is  not  relevant  for  determining 
whether  Pt538  has  had  a  splenectomy. 

(Enter  another  factor  If  you  wish.) 

* «  DON  KNOW 

It  remains  to  collect  further  evidence  for  determining  whether  Pt538 
has  had  a  splenectomy. 


Figure  8.6.  A  Student's  Support  for  an  Incorrect  Hypothesis 

it  turns  out  that  the  student  based  his  hypothesis  on  a  false  analogy  that  he 
derived  from  a  relation  he  had  previously  learned,  but  misunderstood.  The  student 
"knew"  that  a  compromised  host  was  a  patient  whose  immunological  system  was 
impaired.  For  example,  a  drug  that  the  patient  was  receiving  (alcohol,  steroids)  was 
affecting  his  ability  to  fight  an  infection.  The  student  also  knew  that  a  splenectomy 
was  an  operation  in  which  the  spleen  is  'removed.  At  this  point,  interesting 
connections  with  other  facts  could  be  derived  (see  Figure  8.6). 
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Valid  Infarancati 


Faulty  Analogy  t 


COMPROMISED 


SICKLE-CELL-DISEASE 
Causes 

"clogged  spleen" 

(observed) 


DIPLOCOCCUS-PNEUMONIAE 
(weak  CF) 


(hypothesized) 


■sane  effect" 
(assumed) 


SPLENECTOMY 


(hypothesized  strong  CF) 


removed 

spleen 


Figure  6.6.  Student's  Justification  for  an  Hypothesis 


The  student  knew  that  the  effect  of  sickle  celt  disease  was  to  impair  the 
spleen,  leading  to  infection  caused  by  diplococcus  (see  Section  S.2.4.6  for 
discussion).  Reasoning  by  analogy,  he  assumed  that  however  impaired  spleen  function 
affected  the  body,  the  same  effect  was  to  be  expected  if  the  spleen  were  actually 
removed.  He  concluded  that  a  patient  with  no  spleen  at  all  had  an  even  greater 
chance  of  infection  by  diplococcus  than  a  patient  whose  spleen  was  merely  impaired. 
This  was  the  student's  first  mistake,  as  we  will  see  In  a  moment.  MYCIN  does  have  a 
rule  that  says  "if  splenectomy  then  prescribe  therapy  for  diplococcus, "  but  the 
certainty  Is  even  weaker  than  In  the  case  of  sickle  cell  disease.  If  the  student  knew 
why,  he  probably  would  not  have  gone  on. 


Next,  having  considered  one  way  In  which  sickle  cell  disease  and  a  splenectomy 
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were  similar,  the  student  recalled  a  rule  that  concluded  that  sickle  cell  disease  was 
not  present.  This  would  be  useful  because  he  wanted  to  give  GUIDON  evidence  that 
the  patient  had  not  had  a  splenectomy.  The  rule  stated,  "if  the  patient  is  not  a 
compromised  host,  then  he  does  not  have  sickle  cell  disease"  (and  similarly,  if  he  has 
sickle  cell  disease  then  he  is  compromised).  Prom  here  it  was  a  simple  leap  by 
analogy  to  assume  that  a  patient  with  a  splenectomy  would  be  a  compromised  host. 
And  evidence  that  the  patient  was  not  a  compromised  host,  would  be  evidence  that  he 
had  not  had  a  splenectomy.  This  is  what  the  student  told  GUIDON,  but  it  is  wrong. 

The  student's  analogies  are  faulty  because  they  are  based  on  an  erroneous 
justification  for  the  compromised/sickle-cell  relation.  To  see  why,  consider  another 
(correct)  rule  that  concludes  that  sickle  cell  disease  is  not  present:  If  the  patient  is 
not  Black,  then  he  does  not  have  sickle  cell  disease.  It  would  be  ludicrous  to  assume 
that  the  patient  had  not  had  a  splenectomy  because  he  is  not  Black.  When  we  say 
that  an  analogous  relation  between  rules  holds,  we  mean  that  underlying  justification 
for  one  rule  carries  over  to  the  situation  of  another  rule.  Since  the  Black/SIckle-coi' 
rule  is  based  on  a  genetic  argument  and  splenectomy  is  an  operation  that  can  happen 
to  anyone,  there  is  no  analogy  to  be  drawn. 

However,  the  student  thought  that  his  justification  for  the  compromised/sickle' 
cell  rule  would  hold  in  the  case  of  a  splenectomy  as  well.  In  particular,  he  thought 
that  saying  a  sickle-cell  patient  was  compromised  was  merely  restating  the  fact  that 
he  was  likely  to  come  down  with  a  diplococcus-caused  infection.  Indeed,  an  impaired 
spleen  Is  a  form  of  Impairment  of  the  Immunological  system.  This  reasoning 
successfully  carries  over  to  the  case  of  a  splenectomy  because  a  removed  spleen  is 
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assumed  to  simulate  the  conditions  of  an  impaired  spleen.  In  fact,  this  analogy  Is 
perfectly  correct,  It  is  Just  based  on  the  wrong  justification  for  the 
compromised/sickte-cell  rule. 

To  make  a  correct  analogy  with  the  compromised/sickle-cell  rule,  the  student 
has  to  know  the  basis  for  that  rule.  Saying  that  the  rule  merely  "restates"  the  fact 
that  diplococcus  infection  is  likely  is  not  correct.  The  student  did  not  realize  that 
sickle-cell  disease  does  more  to  the  body  than  Impair  the  spleen.  Indeed,  there  are 
several  effects  that  as  whole  severely  impair  the  patient's  health.  These  effects  are 
not  localized  to  the  spleen,  as  in  the  case  of  a  splenectomy.  Therefore,  a 
splenectomy  does  not  fulfill  all  of  the  conditions  that  Justify  the  compromised/sickle¬ 
cell  rule,  and  no  analogy  can  be  drawn. 

ft  is  interesting  to  observe  that  GUIDON  could  theoretically  find  the  relationship 
in  MYCIN's  rules  that  ties  together  compromised  host,  sickle-cell  disease,  diplococcus, 
and  splenectomy.  However,  the  best  it  could  do  would  be  to  hypothesize  a  false 
analogy  between  sickle-cell  disease  and  e  splenectomy.  The  program  knows  nothing 
about  the  spleen,  so  it  cannot  begin  to  diagnose  the  cause  of  the  false  analogy  (over¬ 
simplified  understanding  of  sickle-cell  disease). 

We  do  not  know  if  the  confusion  illustrated  here  is  typical.  It  seems  plausible 
that  this  is  not  an  unusually  complex  reasoning  error.  And  yet,  it  was  surprisingly 
difficult  to  track  down  the  misconception  and  state  Its  origin  succinctly.  Indeed,  the 
student's  mention  of  compromised  host  as  a  factor  to  support  his  hypothesis  was 
originally  only  a  hunch,  based  on  a  vague  recollection  that  there  was  some  underlying 
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connection  between  a  splenectomy  and  likelihood  of  infection.  From  here  we  can  only 
speculate  about  revisions  to  GUIDON  that  will  enable  it  to  detect  a  bug  like  this  and 
articulate  the  confusion  for  the  student.  It  seems  likely  that  a  program  with  this 
capability  will  need  to  question  the  student  about  his  reasoning,  probably  attempting 
to  fit  his  rationale  Into  a  model  of  how  bugs  are  formed,  as  our  example  illustrates  a 
false  analogy  based  on  insufficient  conditions  for  the  basis  relation.  Stevens  and 
Collins  have  made  a  beginning  in  this  kind  of  analysis  [Stevens,  Collins,  &  Goldin. 
1 978].  In  any  case,  to  discuss  underlying  errors  with  the  student,  to  say  more  than 
"compromised  is  irrelevant,"  GUIDON  will  need  a  grasp  on  the  principles  that  support 
MYCIN's  rules.  The  representation  and  acquisition  of  this  knowledge  are  open 
research  problems. 

In  more  general  terms,  we  should  question  just  what  modeling  performance  can 
be  expected  of  a  program  that  does  not  have  voice  and  visual  clues  for  following  a 
student's  thinking.  How  well  do  people  do  at  this  task?  What  resources  do  they  use? 
What  triggers  revision  of  their  model  of  the  student?  What  kinds  of  assumptions  do 
'  they  make  and  how  do  they  test  them?  Building  ICAI  programs  like  GUIDON  may 
involve  having  a  better  understanding  of  expert  human  cognition— a  research 
perspective  we  will  return  to  in  the  final  chapter. 
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GUIDON'S  design  permits  us  to  substitute  a  MYCIN-like  knowledge  base  different 
from  the  infectious  disease  rule  set  used  to  develop  the  initial  tutoring  capabilities. 
This  experiment  was  performed  using  the  PUFF  [Kunz,  et  al.,  1978]  consultation 
system  for  diagnosis  of  pulmonary  function  and  the  SACON  [Bennett,  et  al.,  1978] 
system  for  suggesting  a  structural  analysis  regime.  By  using  GUIDON'S  t-rules  for 
managing  dialogues  Involving  other  rule  sets,  we  hoped  to  demonstrate  that  the  t- 
rules  were  in  fact  independent  of  the  infectious  disease  rule  set,  and  to  shed  light  on 
the  generality  of  the  current  set  of  t-rules.  Aside  from  revealing  a  few  simple 
oversights  in  the  code,  these  experiments  revealed  that  the  t-rules  are  general 
enough,  but  GUIDON  needs  more  strategies  for  economical  discussion  of  complex 
cases. 

Coupling  GUIDON  to  another  rule  set  is  straightforward.  About  an  hour  of 
programming  time  is  required  to  set  up  the  proper  files  and  pointers  in  the  system. 
GUIDON  needs  to  be  told  three  facts  about  the  consultation  system:  its  name  (PUFF, 
SACON);  the  name  given  to  the  object  that  a  case  is  about  (patient,  structure);  and 
the  goal  of  the  consultation  (PULMONARY-DIAGNOSIS,  ANALYSIS-REGIME).  Furthermore, 
the  names  of  the  rule-model  file,  dictionary,  and  parameter  and  rule  annotations  files 
must  be  spec  ified.  GUIDON  is  then  constructed  In  batch  mode,  using  a  modified  (small) 
INTERLISP  core  image,  the  EMYCIN  ( Essential  MYCIN)  package  (rule  interpreter,  user 
Interaction  package,  utility  functions),  tutorial  modules,  and  the  expert  knowledge 
base  (parameters,  rules,  and  domain  variables).  As  usual,  discussic  i  of  a  case  must 
be  preceded  by  a  consultation  and  analysis  that  produces  the  AND/OR  solution  tree. 


This  is  also  done  in  batch  mode. 
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It  should  be  remembered  that  no  additions  were  made  to  these  knowledge  bases 
for  the  purpose  of  tutoring,  in  particular,  the  student  model  was  effectively  disabled 
because  all  rules  were  assumed  to  be  of  expert  level  sophistication  Section  8.2.1. 
Since  the  student  was  a  complete  novice,  and  there  was  no  history  of  previous 
interactions  with  him,  presentation  methods  were  always  selected  that  assumed  that 
the  student  was  ignorant  of  the  domain  rules.  This  tended  to  make  the  tutorial 
considerably  more  verbose  since  step-by-step  discussion  of  rules  was  used  rather 
than  a  direct  quiz.  On  the  other  hand,  the  opportunistic  interruption  strategy  requires 
that  n  rule  used  for  a  quiz  be  within  two  levels  of  the  student's  sophistication;  the 
result  was  that  no  interruptions  were  ever  made. 

Other  knowledge  base  additions  that  we  know  would  be  useful  are  parameter 
definitions  and  rule  justifications.  However,  unlike  MYCIN,  rule  schemas  are  not 
necessary  in  either  PUFF  or  SACON  for  simplifying  summaries  of  evidence.  Rule  models 
were  available  and  appear  in  the  tutorials. 

9.1  The  PUFF  Tutorial 

The  PUFF  knowledge  base  Is  much  simpler  than  MYCIN’s.  There  are  only  60  rules 
and  only  one  "object"  in  a  case,  the  patient.  The  purpose  of  a  PUFF  consultation  is  to 
diagnose  lung  disease,  e.g.,  bronchitis,  asthma. 

The  typescript  has  been  considerably  shortened,  but  otherwise  appears  here  in 
its  original  form.  Omitted  lines  are  indicated  by  annotations. 


» 
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March  30,  1979  10:36PM  in  <CLANCEY>GUIDON.SAV;90330  by  CLANCEY 
This  is  the  tutor  for  PUFF. 


{...22...UNES  ...OMITTED-.} 
Tutorial  Session  30-Mar-79  22:36:48 
Which  case? 

a  a  7 


{...8...UNES  ...OMITTED...} 
[consultation  of  23-Feb-77  1 1 :65AM] 
Initial  Case  Data 


PATIENT- 7: 

The  patient  number  of  PATIENT-7  is  7446. 

Asthma  is  one  of  referral  diagnoses  of  the  patient. 


{...50...LINES...OMITTED...  After  general  orientation,  the  program  delves  into 
the  goal  rule....} 

(Now  considering  RULE001.) 

We  must  now  determine  whether  an  attempt  has  been  made  to  deduce  whether 
there  is  an  interpretation  of  potential  obstructive  airways  disease. 

Let's  consider  a  way  to  determine  this  by  definition.... 

(Now  considering  RULE002.) 

We  must  now  determine  whether  an  attempt  has  been  made  to  deduce  the 
severity  of  obstructive  airways  disease  of  PATIENT-7. 

{...4...P AGES. -OMITTED-  A  major  topic  is  introduced—] 
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We  must  now  determine  whether  an  attempt  has  been  made  to  deduce  the  subtype 
of  obstructive  airways  disease. 

You  should  now  proceed  to  ask  questions  that  will  enable  you  to  make  an 
hypothesis  about  the  subtype  of  obstructive  airways  disease. 


PATIENT- 7  is  a  person.  In  this  context,  when  we  are  considering  the 
subtype  of  obstructive  elrways  disease,  we  generally  find  It  useful  to 
consider: 

1 6a.  the  severity  of  obstructive  airways  disease  of  the  patient 
1 6b.  the  subtype  of  obstructive  airways  disease 
1 6c.  The  degree  of  diffusion  defect  of  the  patient 

There  are  1 1  rules  used  by  PUFF  to  conclude  about  this  topic. 

Altogether  there  are  10  factors  considered  In  these  rules. 

The  values  for  the  subtype  of  obstructive  airways  disease  are:  asthma, 
bronchitis,  emphysema  and  mixed. 

o-  PENDING 

We  need  to  discuss  the  following  subgoals  before  our  discussion  of  the 
subtype  of  obstructive  airways  disease  will  be  complete: 

1  7a.  the  degree  of  lung  restriction  of  PATIENT-7 
1 7b.  The  degree  of  diffusion  defect  of  PATIENT-7 
1  7c.  The  degree  of  reversibility  of  airway  obstruction  of  PATIENT-7 
1 7d.  whether  there  are  post  bronchodilation  test  results 


We  need  to  discuss  the  following  case  data  that  is  directly  relevant  to  the 
subtype  of  obstructive  airways  disease: 

18a.  the  severity  of  coughing  of  PATIENT-7 
18b.  THE  DLCO-Obs/DLCO-pred  ratio  of  PATIENT- 7 
18c.  the  TLC(0LC0)ob8erved/predicted  of  PATIENT- 7 
18d.  the  TLC(body  box)observed/predicted  of  PATIENT-7 


{Tht  usual  options  art  available) 

**  USE  18A 

The  severity  of  coughing  of  PATIENT- 7  is  none. 

If  the  severity  of  coughing  of  PATIENT-7  were  greater  than  or  equal  to 
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mild,  one  could  conclude  that  the  subtype  of  obstructive  airways  disease  is 
bronchitis  (.3)  (other  information  would  also  need  to  be  available  to  make 
this  conclusion)  [RULE047], 

**  DISCUSS  1 7C 

You  should  now  proceed  to  ask  questions  that  will  enable  you  to  make  an 
hypothesis  about  The  degree  of  reversibility  of  airway  obstruction  of 
PATIENT-7. 

*•  FACTORS 

The  following  factors  will  be  useful  to  determine  The  degree  of 
reversibility  of  airway  obstruction  of  PATIENT-7: 

1 9a.  whether  there  are  post  bronchodilation  test  results 
19b.  The  change  in  FEV1  due  to  dilation  of  PATIENT-7 
1 9c.  The  change  In  Resistance  due  to  dilation  of  PATIENT-7 


{In  a  strange  domain,  the  student  might  not  even  know  what  parameters  need  to  be 
deduced  and  which  are  available  simply  by  asking.) 

**  19A 

It  is  not  appropriate  for  the  consultant  to  ask  about  whether  there  are  post 
bronchodilation  test  results. 

You'll  have  to  work  this  out  for  yourself.  Try  the  DISCUSS  option  for 
further  assistance  on  this  topic. 

«*  19B 

The  change  In  FEV1  due  to  dilation  of  PATIENT-7  Is  31. 

««  19C 

The  change  in  Resistance  due  to  dilation  of  PATIENT-7  is  20. 

>>>  20a.  PUFF  has  made  a  FINAL  CONCLUSION  about  whether  there  are  post 
bronchodilation  test  results. 

>>>  21a.  PUFF  just  made  a  conclusion  about  The  degree  of  reversibility  of 
airway  obstruction  of  PATIENT- 7. 

»>  22a.  PUFF  just  made  a  conclusion  about  the  findings  about  the  diagnosis 
of  obstructive  airways  disease. 

»>  23a.  PUFF  just  made  a  conclusion  about  the  findings  about  the  diagnosis 
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of  obstructive  airways  disease. 

We  have  substantially  completed  our  discussion  of  The  degree  of 
reversibility  of  airway  obstruction  of  PATJENT-7. 

Would  you  like  to  make  a  hypothesis  now  (as  opposed  to  gathering  more 
information)? 

**  NO 


{ Opportunistic  tutoring:  see  Chapter  /  for  discussion  of  this  part  of  the  dialogue .} 


We  will  now  consider  evidence  that  leads  to  a  conclusion  about  The  degree  of 
reversibility  of  airway  obstruction  of  PATIENT-7. 

Here  are  some  conclusions  that  you  may  not  have  considered.... 

The  change  in  FEV1  due  to  dilation  of  PATIENT-7  and  The  change  in  Resistance 
due  to  dilation  of  PATIENT-7  Is  evidence  that  there  are  post  bronchodilation 
test  results  [RULE028J. 


Given  the  fact  that  there  are  post  bronchodilation  test  results  and  The 
change  in  FEV1  due  to  dilation  of  PATIENT-7,  does  this  tell  you  that  The 
degree  of  reversibility  of  airway  obstruction  of  PATIENT-7  is  good  (.5)? 

**  DONTKNOW 

Given  these  facts,  we  can  conclude  that  The  degree  of  reversibility  of 
airway  obstruction  of  PATIENT-7  is  significant  (.8)  [RULE020]. 

The  conclusions  in  the  question  hold  when  The  change  in  FEV1  due  to  dilation 
of  the  patient  Is  between  6  and  10. 


»  TOPIC 

We  are  discussing  The  degree  of  reversibility  of  airway  obstruction  of 
PATIENT-7. 

Hypotheses  you  should  be  considering  are:  none,  insignificant,  slight,  good 
and  significant. 


{ GUIDON'S  phrasing  Is  awkward  for  values  like  ’none"  and  * significant, ’  but  the 
reasoning  is  valid.) 

••  HYPOTHESIS 

What  is  The  degree  of  reversibility  of  airway  obstruction  of  PATIENT- 7? 


♦ 
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«•  SIGNIFICANT 

You're  right,  there's  good  reason  to  suspect  significant. 

The  fact  that  there  are  post  bronchodilation  test  results  and  The  change  in 
FEV1  due  to  dilation  of  PATIENT- 7  is  evidence  that  The  degree  of 
reversibility  of  airway  obstruction  of  PATIENT-7  is  significant  (.8) 
[RULE020]. 


{...3... PACES... OMITTED] 

*  *  *  *  *  *  •  *  • 


Back  to  our  discussion  of  the  findings  about  the  diagnosis  of  obstructive 
airways  disease.... 

**  USE  350 

The  number  of  pack-years  smoked  Is  1 7. 

>>>  37a.  PUFF  just  made  a  conclusion  about  the  findings  about  the  diagnosis 
of  obstructive  airways  disease. 

Using  other  information  available  to  you  now,  the  number  of  pack-years 
smoked  enables  you  to  conclude  that  Airway  obstruction  is  consistent  with 
the  patient's  smoking  history,  is  one  of  the  findings  about  the  diagnosis 
of  obstructive  airways  disease  [RULEO30]. 

**  JUSTIFY  RULE036 

RULEINFOFILE  does  not  exist 


{Again,  the  hint  makes  sense,  hut  it  could  be  worded  better.} 

"  HINT 

Hint:  Look  for  strong,  positive  evidence  that  Elevated  lung  volumes  indicate 
overinflation.  Is  one  of  the  findings  about  the  diagnosis  of  obstructive 
airways  disease. 


{...49. ..LINES.. OMITTED...} 

We  have  substantially  completed  our  discussion  of  the  findings  about  the 
diagnosis  of  obstructive  airways  disease. 

Would  you  like  to  make  a  hypothesis  now  (as  opposed  to  gathering  more 
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information)? 

«  YES 


{The  student  must  enter  internal  identifiers  for  the  text-valued  parameter.) 

What  are  the  findings  about  the  diagnosis  of  obstructive  airways  disease? 

*"  SOVERINFL  (9)  SSMOKE  (5) 

You're  right,  there's  strong  evidence  for  $overinfl  and  reason  to  suspect 
$smoke. 

We  can  conclude  that  the  findings  about  the  diagnosis  of  obstructive  airways 
disease  are: 

Soverinfl  (considering  the  RV/RV-predicted  ratio  of  PATIENT*?  and 
the  TLC/TLC-predlcted  ratio  of  PATIENT-7  [RULE005]), 

Ssmoke  (considering  the  severity  of  obstructive  airways  disease  of 
PATIENT-7  and  the  number  of  pack-years  smoked  [RULE036]). 

What  facts  about  this  case  tell  you  that  the  findings  about  the  diagnosis  of 
obstructive  airways  disease  are:  $asthma2  (1.0)  Sasthmal  (1.0)  $rev4  (1.0) 
$dilation  (1.0)  $mmf  (1.0)  $fev1/fvc1  (1.0)  $f6026  (1.0)? 

(Enter  one  factor  per  line.) 

«»  HELP 


AGES. ..OMITTED...  Discussing  "summary  statements "  poses  problems 
because  it,  like  "findings,"  is  really  a  cluster  of  simple  truelfalse  parameters  that 
would  have  distinct  labels  in  the  Infectious  disease  system.) 

We  must  now  determine  whether  an  attempt  has  been  made  to  deduce  the  summary 
statements  about  this  interpretation. 

Try  to  make  a  conclusion  about  the  summary  statements  about  this 
interpretation. 

What  are  the  summary  statements  about  this  interpretation? 

HELP 

What  facts  about  this  case  tell  you  that  Obstructive  Airways  Disease  of  the 
asthmatic  type  is  one  of  the  summary  statements  about  this  interpretation 
(.61)? 

(Enter  one  factor  per  line.) 

•»  350 

Referral  diagnoses  of  PATIENT-7  can  be  used  to  determine  the  subtype  of 
obstructive  airways  disease... 

and  this  enables  us  to  determine  the  summary  statsments  about  this 
interpretation. 
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*«  35G 

The  degree  of  reversibility  of  airway  obstruction  of  PATIENT-7  can  be 
used  to  determine  the  subtype  of  obstructive  airways  disease... 
and  this  enables  us  to  determine  the  summary  statements  about  this 
interpretation. 


You're  right  about  the  subtype  of  obstructive  airways  disease,  however  you 
should  have  mentioned  the  severity  of  obstructive  airways  disease  of 
PATIENT- 7. 


(Continuing  our  discussion  of  the  interpretation  [RULE001].) 

We  have  already  determined  that  an  attempt  has  been  made  to  deduce  whether 
the  test  was  done  properly. 

We  have  already  determined  that  an  attempt  has  been  made  to  deduce  whether 
there  are  inconsistencies  in  the  data. 

At  this  point,  PUFF  would  print  the  interpretation  of  the  pulmonary  function 
tests. 

(End  of  our  discussion  of  RULE001) 


This  completes  our  discussion  of  PATIENT-7. 
Saving  student  model...done. 

Do  you  have  any  comments? 

**  NO 

Thank  you. 


0.2  Discussion  of  the  PUFF  Tutorial 


The  PUFF  tutorial  demonstrates  that  GUIDON'S  t-rules  are  in  fact  independent  of 
the  rule  set  of  the  expert  system.  The  author  played  the  role  of  student;  his 
background  was  insufficient  for  solving  the  problem  without  assistance.  The  subgoals 
list  was  useful  for  getting  some  grasp  of  the  structure  of  the  rule  set,  though  an 
explanation  of  the  flow  volume  loop  model  of  pulmonary  function  would  have  been  more 
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useful.  It  was  hard  to  remember  the  meaning  of  abbreviations  like  FEV1/FVC.  PUFF 
does  not  have  either  a  definition  file  or  justifications  for  its  rules. 

A  PUFF  finding  is  a  text  string  that  is  printed  after  the  consultation,  constituting 
the  interpretation  of  the  case,  e.g.,  "Good  response  to  bronchodilators  is  consistent 
with  asthmatic  condition,  and  their  continued  use  is  indicated."  Unfortunately,  GUIDON 
does  not  give  a  text-string-valued  parameter  the  special  treatment  It  requires,  so  "is 
one  of  the  findings  about  the  diagnosis  of  obstructive  airways  disease"  is  tacked  on 
to  statements  unnecessarily.  More  seriously,  the  hypothesis  evaluation  and  question¬ 
asking  routines  expect  the  student  to  type  In  the  internal  identifier  for  findings 
($smoke,  $overinfl).  Clearly,  some  form  of  parser  Is  required  here. 

The  careful  reader  will  observe  that  the  program  says,  "PATIENT- 1  Is  a  person" 
(just  as  in  SACON  it  will  say,  "STRUCTURE-3934  is  a  structure").  When  an  object  in 
the  context  tree  need  not  be  qualified  (as  a  culture  can  be  pending,  positive,  or 
negative),  then  this  description  can  be  left  off. 

The  most  serious  theoretical  problem  that  we  observe  is  the  tutorial's  verbosity. 
When  most  of  the  parameters  are  determined  by  single  rules,  as  hi  PUFF,  the  tutor 
methodically  talks  the  student  through  the  case,  goal  by  goal,  providing  infrequent 
opportunity  for  student  initiative.  (Though,  in  part  this  is  exacerbated  in  the  PUFF  and 
SACON  tutorials  by  assuming  that  the  student  knows  nothing.)  GUIDON  appears  to  do  a 
decent  job  of  generating  coherent  text  by  traversing  the  AND/OR  tree,  but  has  no 
awareness  that  dozens  of  "economical"  presentations  are  adding  up  to  a  two  page 
monologue.  We  saw  this  problem  at  the  end  of  of  the  MYCIN  protocol,  and  it  recurs  in 


more  extreme  form  in  SACON. 
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8.3  The  SACON  Tutorial 

SACON's  rule  set  is  comparable  to  MYCIN’s  in  size,  and  this  case  (for  the 

"College  Union  Building")  is  more  than  3  times  the  size  of  any  tutorial  previously  run  on 

GUIDON  (in  terms  of  the  amount  of  case  data  and  number  of  rule  invocations). 

April  7,  1979  7:49PM  In  <CLANCEY>GUIDON.SAVi90407  by  CLANCEY 

This  is  the  tutor  for  SACON. 

Tutorial  Session  7-Apr-79  19:48:40 

Which  case? 

««  3934 


{...6...LINES...OMITTED...  Observe  the  large  amount  of  initial  information 
describing  the  structure  to  be  analyzed—) 

[consultation  of  7-Jul-78  4:46PM] 

Initial  Case  Data 


STRUCTURE-3934: 

The  name  of  STRUCTURE-3934  is  college  union  building. 

The  analysis  error  (in  percent)  that  Is  tolerable  is  10. 

Both  is  one  of  the  integrity  evaluation  goals  of  the  analysis. 

The  college  union  building  does  not  have  time  dependent  terms  in 
its  equations  of  equilibrium. 

The  college  union  building  does  not  have  temperature  dependent 
terms  in  its  equations  of  equilibrium. 

There  were  2  sub-structures,  as  follows... 

SUB-STRUCTURE- 1: 

The  name  of  SUB-STRUCTURE-1  Is  floor  section. 

The  material  composing  the  floor  section  (SUB¬ 
STRUCTURE-1  )  is  concrete. 

The  length  of  the  floor  section  (SUB-STRUCTURE- 1)  is 
384  Inches. 

The  weight  of  the  floor  section  (SUB-STRUCTURE- 1)  is 
108800  pounds. 

The  floor  section  (SUB-STRUCTURE-1)  is  solid. 

The  configuration  of  support  for  the  floor  section 
(SUB-STRUCTURE-1)  is  four-sides. 

There  was  1  loadings  of  ths  sub-structure,  as 
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follows... 

LOADING- 1: 

The  name  of  LOADING- 1  is  total  loading. 

The  number  of  cycles  LOADING- 1  la  to  be  applied 
is  1000. 

There  was  1  loading  components  of  the  loading, 
as  follows... 

LOAD-COMPONENT-1 : 

The  distribution  of  LOAD- 
COMPONENT- 1  is  distributed. 

The  surface  to  which  LOAD- 
COMPONENT- 1  acts  normal  is 
width-length. 


{...S8...LINES... OMITTED...) 


[Now  giving  initial  case  data  to  SACON...  this  may  take  a  few  minutes.] 


>>>  2a.  SACON  just  made  a  conclusion  about  the  deflection  phenomena  in  the 
north-south  columns  (SUB-STRUCTURE-2). 

>>>  3a.  SACON  just  made  a  conclusion  about  the  deflection  phenomena  in  the 
floor  section  (SUB-STRUCTURE- 1 ). 

>»  4a.  SACON  has  made  a  FINAL  CONCLUSION  about  the  stress  criterion  of  the 
floor  section  (SUB-STRUCTURE- 1 ). 

>>>  6a.  SACON  has  made  a  FINAL  CONCLUSION  about  Young's  modulus  of  the 
material. 

>>>  6a.  SACON  has  made  a  FINAL  CONCLUSION  about  the  density  of  the 
material. 

»>  7a.  SACON  has  made  a  FINAL  CONCLUSION  about  the  stress  criterion  of  the 
north-south  columns  (SUB-STRUCTURE-2). 

>>>  8a.  SACON  has  made  a  FINAL  CONCLUSION  about  Young's  modulus  of  the 
material. 

»>  9a.  SACON  has  made  a  FINAL  CONCLUSION  about  the  density  of  the 
material. 

[SACON  is  now  considering  additional  questions  to  ask....] 
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We  will  now  begin  to  discuss  this  case,  limiting  our  discussion  to  one  topic 
at  a  time. 

In  order  to  determine  the  analysis  strategy  of  the  college  union  building  we 
will  want  to  consider: 

1 )  the  analysis  class  of  the  college  union  building,  and 

2)  the  analysis  recommendations  to  be  considered  when  preparing  the 
college  union  building  for  modelling 


{...64...LI  N  ES...OM  ITT  ED...  The  goal  rule  has  been  started;  orientation  is  given.) 


(Now  considering  RULE001.) 

We  must  now  determine  whether  the  analysis  class  of  the  college  union 
building  is  known. 

You  should  now  proceed  to  ask  questions  that  will  enable  you  to  make  an 
hypothesis  about  the  analysis  class  of  the  college  union  building. 

STRUCTURE-3934  is  a  structure.  In  this  context,  when  we  are 
considering  the  analysis  class  of  the  college  union  building,  we  generally 
find  it  useful  to  consider: 

11a.  the  types  of  nonlinearity  in  the  structure 
1 1  b.  the  deflection  phenomena  in  the  structure 
11c.  the  stress  behaviour  phenomena  in  the  structure 

There  are  36  rules  used  by  SACON  to  conclude  about  this  topic. 

Altogether  there  are  3  factors  considered  in  these  rules. 

Some  sample  values  for  the  analysis  class  of  the  structure  are:  nonlinear- 
geometry-strength,  inelastic-stiffness-degradation,  inelastic-stress-failure 
and-  nonlinear-stress-margin,  etc. 

•«  FACTORS 

The  following  factors  will  be  useful  to  determine  the  analysis  class  of  the 
college  union  building: 

1 2a.  the  types  of  nonlinearity  In  the  college  union  building 
1 2b.  whether  the  support  conditions  of  the  college  union  building 
are  nonlinear 

12c.  the  non-dimensional  deflection  of  the  floor  section  (SUB¬ 
STRUCTURE-1) 
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**  DISCUSS  12A 

There  are  no  rules  which  successfully  apply  to  this  case  that  can  tell  us 
about  the  types  of  nonlinearity  In  the  college  union  building. 


>>>  13a.  SACON  has  made  a  FINAL  CONCLUSION  about  the  types  of  nonlinearity 
in  the  college  union  building. 

Back  to  our  discussion  of  the  analysis  class  of  the  college  union 
building.... 


{GUIDON  proceeds  to  delve  Into  the  tree,  introducing  three  topics  in  succession, 
each  a  level  deeper,  and  each  concluded  by  a  single  rule  that  is  presented  clause- 
by-clause.} 

**  DISCUSS  12C 

Let's  consider  a  way  to  determine  this  by  definition.... 


(Now  considering  RULE098.) 

We  must  now  determine  whether  you  have  examined  the  loadings  of  the  sub¬ 
structure,  selecting  those  for  which  the  maximum  deflection  bound  at  a 
point,  due  to  all  components  of  this  loading  is  known. 

Let's  consider  a  way  to  determine  this  by  definition.... 


(Now  considering  RULE1 12.) 

We  must  now  determine  whether  you  have  examined  the  loading  components  of 
the  loading,  selecting  those  for  which  the  deflection  magnitude  of  this  load 
component  in  inches  Is  known. 

Let's  consider  a  way  to  determine  this  by  definition.... 


(Now  considering  RULE168.) 

We  have  already  determined  that  the  surface  to  which  LOAD-COMPONENT- 1  acts 
normal  is  width-length. 

We  have  already  determined  that  the  distribution  of  LOAD-COMPONENT- 1  is 
distributed. 

We  have  already  determined  that  the  configuration  of  support  for  the  floor 
section  (SUB-STRUCTURE-1)  is  four-sides. 

We  must  now  determine  whether  the  shape  of  the  floor  section 


» 
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(SUB-STRUCTURE-1)  is  one  of:  the  surface  shapes. 


{ A  definitional  rule  is  presented  here  by  giving  new  case  data  and  quizzing.} 

Here  is  some  relevant  data  you  could  have  asked  for:  the  geometry  of  the 
floor  section  (SUB-STRUCTURE- 1)  is  planar,  the  modelling  dimensionality  of 
the  floor  section  (SUB-STRUCTURE- 1)  is  2,  and  continuum  is  one  of  the 
constructions  of  the  floor  section  (SUB-STRUCTURE-1). 

What  does  this  tell  you  about  the  shape  of  the  floor  section  (SUB¬ 
STRUCTURE-1  )? 

*«  HELP 

Can  you  conclude  that  the  shape  of  the  floor  section 
(SUB-STRUCTURE- 1)  is 
(select  from  the  following): 

1 )  beam  —  suggestive  evidence  (.62) 

2)  semimonocoque  —  suggestive  evidence  (.62) 

3)  not  shell  —  strongly  suggestive  evidence  (-.95) 

4)  plate  —  with  certainty  (1 .0) 

XX  /J 


Yes,  that's  right  (see  RULE  100). 


>>>  1 4a.  SACON  has  made  a  FINAL  CONCLUSION  about  the  shape  of  the  floor 
section  (SUB-STRUCTURE-1). 

(Continuing  our  discussion  of  the  deflection  magnitude  of  LOAD-COMPONENT- 1 
in  Inches  [RULE  158].) 


{Clause-by-clause  treatment  of  this  lengthy  rule  continues....} 

We  must  now  determine  whether  the  magnitude  of  LOAD-COMPONENT-1  (in  psl)  Is 
known. 

The  magnitude  of  LOAD-COMPONENT- 1  (in  psl)  is  information  about  a  case  that 
is  usually  available  to  the  consultant. 

The  magnitude  of  LOAD-COMPONENT- 1  (in  psl)  is  1.3. 


(Continuing  our  discussion  of  the  deflection  magnitude  of  LOAD-COMPONENT- 1 
In  inches  [RULE  1 68] ) 

We  have  already  determined  that  the  length  of  the  floor  section  (SUB¬ 
STRUCTURE-1)  Is  known. 
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We  must  now  determine  whether  the  depth  of  the  floor  section 
(SUB-STRUCTURE-1)  is  known. 

The  depth  of  the  floor  section  (SUB-STRUCTURE- 1)  is  Information  about  a  case 
that  is  usually  available  to  the  consultant. 


{This  single  piece  of  information  is  the  key  to  many  rules.  Observe  how  many 
conclusions  are  now  made....} 

The  depth  of  the  floor  section  (SUB-STRUCTURE-1)  is  8.5. 

>>>  1 6a.  SACON  has  made  a  FINAL  CONCLUSION  about  the  moment  of  inertia  of 
the  floor  section  (SUB-STRUCTURE- 1). 

>>>  16a.  SACON  has  made  a  FINAL  CONCLUSION  about  the  effective  width  of  the 
floor  section  (SUB-STRUCTURE- 1). 

>>>  1 7a.  SACON  has  made  a  FINAL  CONCLUSION  about  gamma. 

>>>  16a.  SACON  has  made  a  FINAL  CONCLUSION  about  alpha. 

>>>  19a.  SACON  has  made  a  FINAL  CONCLUSION  about  the  stress  magnitude  of 

LOAD-COMPONENT- 1  (in  psi). 

>>>  20a.  SACON  has  made  a  FINAL  CONCLUSION  about  the  maximum  stress  bound 
at  a  point,  due  to  ali  components  of  LOADING- 1  (in  psi). 

>>>  21a.  SACON  has  made  a  FINAL  CONCLUSION  about  the  non-dimensional  stress 
of  the  floor  section  (SUB-STRUCTURE- 1 ). 

>>>  22a.  SACON  has  made  a  FINAL  CONCLUSION  about  the  stress  behaviour 
phenomena  In  the  floor  section  ( SUB- STRUCTURE- 1 ). 

>>>  23a.  SACON  has  made  a  FINAL  CONCLUSION  about  the  deflection  magnitude 
of  LOAD-COMPONENT- 1  in  inches. 

>>>  24a.  SACON  has  made  a  FINAL  CONCLUSION  about  the  maximum  deflection 
bound  at  a  point,  due  to  all  components  of  LOADING- 1. 

>>>  25a.  SACON  has  made  a  FINAL  CONCLUSION  about  the  non-dimensional 
deflection  of  the  floor  section  (SUB-STRUCTURE- 1). 

>»  26a.  SACON  has  made  a  FINAL  CONCLUSION  about  the  types  of  nonlinearity 
In  the  floor  section  (SUB-STRUCTURE- 1). 

»>  27a.  SACON  has  made  a  FINAL  CONCLUSION  about  the  deflection  phenomena 
in  the  floor  section  (SUB-STRUCTURE- 1 ). 

>>>  28a.  SACON  just  made  a  conclusion  about  the  deflection  phenomena  In  the 
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college  union  building. 


{5ft7/  at  the  deepest  level,  GUIDON  continues  with  rulelJS....} 

(Continuing  our  discussion  of  the  deflection  magnitude  of  LOAD-COMPONENT- 1 
in  inches  [RULE1CS].) 

We  must  now  determine  whether  alpha  is  known. 

The  length  of  the  floor  section  (SUB-STRUCTURE-1)  and  the  effective  width  of 
the  floor  section  (SUB-STRUCTURE- 1)  is  evidence  that  alpha  is  .063 
[RULE  1 07  J. 

(Continuing  our  discussion  of  the  deflection  magnitude  of  LOAD-COMPONENT- 1 
in  inches  [RULE1 58].) 


We  must  now  determine  whether  gamma  is  known. 

The  length  of  the  floor  section  (SUB-STRUCTURE- 1 )  and  the  effective  width  of 
the  floor  section  (SUB-STRUCTURE-1)  is  evidence  that  gamma  Is  .062 
[RULE  109]. 

(Continuing  our  discussion  of  the  deflection  magnitude  of  LOAD-COMPONENT- 1 
in  inches  [RULE  168].) 

We  must  now  determine  whether  the  moment  of  inertia  of  the  floor  section 
(SUB-STRUCTURE- 1)  is  known. 

The  shape  of  the  floor  section  (SUB-STRUCTURE- 1),  the  floor  section  (SUB¬ 
STRUCTURE-1),  the  depth  of  the  floor  section  (SUB-STRUCTURE- 1 ),  the  weight 
of  the  floor  section  (SUB-STRUCTURE-1),  the  density  of  the  material  and  the 
length  of  the  floor  section  (SUB-STRUCTURE-1)  is  evidence  that  the  moment  of 
inertia  of  the  floor  section  (SUB-STRUCTURE- 1 )  is  61  [RULE  106]. 

(Continuing  our  discussion  of  the  deflection  magnitude  of  LOAD-COMPONENT-1 
In  inches  [RULE  158].) 

We  must  now  determine  whether  Young's  modulus  of  the  material  is  known. 

The  material  composing  the  floor  section  (SUB-STRUCTURE- 1 )  is  evidence  that 
Young's  modulus  of  the  material  is  2000000  [RULE  160]. 

(Continuing  our  discussion  of  the  deflection  magnitude  of  LOAD-COMPONENT- 1 
in  inches  [RULE  168].) 

We  can  now  conclude  that  the  deflection  magnitude  of  LOAD-COMPONENT- 1  in 
inches  is  .0073  inches  (considering  the  surface  to  which  LOAD-COMPONENT- 1 
acts  normal,  the  distribution  of  LOAD-COMPONENT-1,  the  configuration  of 
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support  for  the  floor  section  (SUB-STRUCTURE-1),  the  shape  of  the  floor 
section  (SUB-STRUCTURE- 1),  the  magnitude  of  LOAD-COMPONENT- 1  (in  psi),  the 
length  of  the  floor  section  (SUB-STRUCTURE- 1 ),  the  depth  of  the  floor 
section  (SUB-STRUCTURE- 1),  alpha,  gamma,  the  moment  of  inertia  of  the  floor 
section  (SUB-STRUCTURE-1)  and  Young's  modulus  of  the  material  [RULE158]). 

However,  there  are  other  rules  that  we  might  discuss  that  use  information 
you  have  already  been  given. 

(End  of  our  discussion  of  a  rule  to  determine  the  deflection  magnitude  of 
LOAD-COMPONENT- 1  in  Inches  [RULE  168].) 


{Finally,  the  discussion  pops  up  a  level.  Quickly,  the  reasoning  chain  unwinds  to 
the  top.} 

(Continuing  our  discussion  of  the  maximum  deflection  bound  at  a  point,  due 
to  all  components  of  LOADING-1  [RULE1 12].) 

We  can  now  conclude  that  the  maximum  deflection  bound  at  a  point,  due  to  all 
components  of  LOADING- 1  is  .0073  (considering  the  deflection  magnitude  of 
LOAD-COMPONENT- 1  In  inches  [RULE112]). 

However,  there  are  other  rules  that  we  might  discuss  that  use  Information 
you  have  already  been  given. 

(End  of  our  discussion  of  a  rule  to  determine  the  maximum  deflection  bound 
at  a  point,  due  to  all  components  of  LOADING-1  [RULE1 12].) 


(Continuing  our  discussion  of  the  non-dimensional  deflection  of  the  floor 
section  (SUB-STRUCTURE- 1)  [RULE098].) 

We  can  now  conclude  that  the  non-dimensional  deflection  of  the  floor  section 
(SUB-STRUCTURE-1)  Is  .0073  (considering  the  maximum  deflection  bound  at  a 
point,  due  to  all  components  of  LOADING- 1  [RULE098]). 

However,  there  are  other  rules  that  we  might  discuss  that  use  information 
you  have  already  been  given. 

(End  of  our  discussion  of  a  rule  to  determine  the  non-dimensional  deflection 
of  the  floor  section  (SUB-STRUCTURE- 1)  [RULE098].) 


Back  to  our  discussion  of  the  analysis  class  of  the  college  union 
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building... 


{We  are  now  back  to  the  point  where  the  student  asked  to  discuss  the  non- 
dimensional  deflection  of  the  floor  section  (l 2c).) 

••  PENDING 

We  need  to  discuss  the  following  case  data  that  Is  directly  relevant  to  the 
analysis  class  of  the  college  union  building: 

29a.  whether  the  support  conditions  of  the  college  union  building 
are  nonlinear 


**  29A 

The  support  conditions  of  the  college  union  building  are  not  nonlinear. 

We  have  substantially  completed  our  discussion  of  the  analysis  class  of  the 
college  union  building. 

Would  you  like  to  make  a  hypothesis  now  (as  opposed  to  gathering  more 
information)? 

«*  YES 


{ The  student  makes  an  hypothesis.) 

What  are  the  analysis  class  of  the  college  union  building? 

*■  LINEAR-ANALYSIS 

You're  right,  there's  strong  evidence  for  linear-analysis. 

The  fact  that  the  types  of  nonlinearity  in  the  college  union  building  is  not 
known  and  the  fact  that  the  support  conditions  of  the  college  union  building 
are  not  nonlinear  is  evidence  that  linear-analysis  is  one  of  the  analysis 
class  of  the  college  union  building  [RULE003]. 

It  remains  to  collect  further  evidence  for  determining  the  analysis  class  of 
the  college  union  building. 

However,  we  have  substantially  completed  our  discussion  of  the  analysis 
class  of  the  college  union  building. 

Would  you  like  to  end  discussion  of  this  topic  now  (i.e.,  get  the  chance  to 
see  a  summary  of  all  evidence  that  can  eventually  be  collected  instead  of 
working  It  out  yourself)? 
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**  YES 


»>  30a.  SACON  has  made  a  FINAL  CONCLUSION  about  the  analysis  class  of  the 
college  union  building. 

Our  final  conclusion  is  that  linear-analysis  Is  one  of  the  analysis  class  of 
the  college  union  building. 

Would  you  like  to  see  a  summary  of  the  evidence  that  supports  this 
conclusion? 

*»  NO 


ft  ft  ft  RUN  *  *  * 


Our  discussion  of  the  analysis  class  of  the  college  union  building  is 
complete. 


{Discussion  of  the  goal  rule  continues.) 

(Continuing  our  discussion  of  the  analysis  strategy  of  the  college  union 
building  [RULE001].) 

We  must  now  determine  whether  an  attempt  has  been  made  to  deduce  the 
analysis  recommendations  to  be  considered  when  preparing  the  college  union 
building  for  modelling. 

You  should  now  proceed  to  ask  questions  that  will  enable  you  to  make  an 
hypothesis  about  the  analysis  recommendations  to  be  considered  when 
preparing  the  college  union  building  for  modelling. 

There  are  1 8  rules  used  by  SACON  to  conclude  about  this  topic. 

Altogether  there  are  4  factors  considered  in  these  rules. 

Some  sample  values  for  the  analysis  recommendations  to  be  considered  when 
preparing  the  structure  for  modelling  are:  $rec13,  $rec1 1,  $rec14,  $rec10, 
$rec6  and  $rec16,  etc. 


AGES  ...OM ITTED...} 

ft  ft  ft  *  *  *  *  *  * 


i 


Our  discussion  of  the  analysis  recommendations  to  be  considered  when 
preparing  the  college  union  building  for  modelling  is  complete. 
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(Continuing  our  discussion  of  the  analysis  strategy  of  the  college  union 
building  [RULE001J.) 

At  this  point,  SACON  would  Using  the  information  collected  during  the 
consultation,  recommend  an  analysis  method  for  this  structure. 

(End  of  our  discussion  of  RULE001) 

This  completes  our  discussion  of  STRUCTURE-3934. 

Saving  student  modeL.done. 

Do  you  have  any  comments? 

**  NO 

Thank  you 

9.4  Discussion  of  the  SACON  tutorial 

Why  is  the  tutorial  so  verbose?  When  GUIDON  was  developed  using  the 
infectious  disease  rule  set  of  MYCIN,  the  main  problem  was  the  tediousness  of 
requiring  the  student  to  specifically  request  each  piece  of  data.  Methods  were 
developed  for  cutting  short  a  discussion,  acquiring  data  in  blocks  or  all  at  once,  and 
presenting  new  data  to  the  student  in  combination  with  a  quiz.  In  SACON,  however, 
most  of  the  data  are  given  initially  as  part  of  the  problem  definition  (this  is  to  be 
expected  for  well-defined  computational  problems,  In  contrast  to  medical  problems  in 
which  there  are  too  many  world  facts  that  could  possibly  be  relevant  to  list  them  all 
initially).  Consequently,  the  program  rarely  gives  the  student  an  opportunity  to  collect 
more  data,  and  instead  reads  out  the  many  intermediate,  definitional  conclusions.  The 
student  is  not  asked  to  make  an  hypothesis  because  it  is  assumed  that  the  rules  are 
too  sophisticated  for  him.  Indeed,  they  were. 


SACON's  task  is  a  computation-intensive  problem  that  requires  table  look-up  and 
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formula  calculation  (to  find  out  parameters  like  "alpha,"  and  "young's  modulus”). 
Observe  that  the  analysis  class  parameter  in  SACON  involves  36  rules,  but  only  3 
factors.  The  second  major  topic,  analysis  recommendations  has  the  ratio  of  1 8  rules 
to  4  factors.  There  are  many  more  intermediate  subgoals  than  terminal  case  data 
nodes  in  the  tree.  This  explains  why  SACON  suddenly  makes  1 4  conclusions  when  the 
depth  of  the  floor  section  is  known.  Clearly,  this  kind  of  problem  is  poorly  suited  to 
the  kind  of  case  method  dialogue  formalized  in  GUIDON'S  t-rules.  GUIDON  behaves 
better  when  there  Is  a  large  number  of  rules,  using  many  factors.  This  is  why  the 
strategy  for  detecting  substantial  completion  of  a  goal  was  implemented  (Section 
6.2. 1.2).  This  strategy  is  not  needed  in  SACON  because  a  topic  is  finished  after 
asking  just  a  few  questions.  Instead,  for  discussing  SACON  cases,  GUIDON  needs 
methods  for  cutting  down  on  the  number  of  subgoals  that  are  mentioned,  not  just  the 
number  of  rules. 

Leaving  out  rules  is  somewhat  easier  than  leaving  out  subgoals.  When  GUIDON 
omits  discussion  of  a  MYCIN  rule.  It  Is  usually  omitting  just  a  single  factor.  If  the  rule 
fails,  or  contributes  insignificant  evidence,  this  omission  is  usually  of  little  importance. 
However,  to  leave  out  a  subgoal  is  to  omit  a  subtree  of  the  solution,  not  just  one  of 
the  terminal  nodes.  The  current  set  of  t-rules  methodically  sets  up  each  subgoal  for 
formal  discussion.  The  program  needs  methods  for  skipping  over  a  topic  or  mentioning 
it  in  passing. 

To  know  that  a  subtree  can  be  omitted,  the  program  must  reason  more  deeply 
about  a  case.  Current  t-rules  only  look  one  level  ahead  and  choose  a  strategy  to 
discuss  the  next  subgoat  or  next  rule.  In  SACON  the  result  is  a  lengthy,  depth-first 
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tour  through  the  rules.  This  shortcoming  is  quits  similar  to  the  disconnected  dialogue 
shown  In  Section  6.2.4,  In  which  the  procedure  for  giving  advice  after  an  hypothesis 
‘  did  not  look  ahead  to  see  If  the  subproblem  was  substantially  complete.  Managing  the 
dialogue  requires  more  than  clever  methods  for  discussing  a  single  rule  or  a  sinqie 
goal.  The  tutor  must  look  ahead  to  get  s  more  comprehensive  picture  of  the  structure 
of  the  solution. 

In  fact,  this  experiment  has  revealed  a  basic  property  of  rule  sets  that 
determines  what  kinds  of  dialogue  management  problems  will  arise.  In  MYCIN,  the 
solution  tree  tends  to  be  shallow  and  broad,  with  about  as  many  rules  as  terminal  case 
data  nodes.  Pruning  rules  from  the  discussion  is  about  equivalent  to  leaving  out  a 
piece  of  data.  However,  in  SACON  the  number  of  rules  is  much  greater  than  the 
number  of  terminal  case  data  nodes,  and  pruning  rules  must  involve  deliberately  not 
discussing  a  subgoal.  Moreover,  most  subgoals  are  determined  by  a  single  rule,  so  the 
main  strategy  for  shortening  discussion,  detection  of  an  incomplete  subgoal,  is  not 
important.  Instead,  management  of  the  dialogue  requires  omitting  some  topics  entirely. 

In  conclusion,  the  exploratory  experiment  of  coupling  GUIDON  to  other  knowledge 
bases  has  been  very  useful.  In  particular,  it  has  revealed  how  the  basic  structure  of 
a  rule  set  determines  the  kinds  of  dialogue  management  problems  that  GUIDON  must 
address,  showing  that  the  current  set  of  t-rules  is  tuned  to  a  certain  extent  to  the 
shape  of  the  infectious  disease  reasoning  tree. 
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SUMMARV  AND  CONCLUSIONS 


This  chapter  presents  a  summary  of  the  research  accomplishments  of  this 
dissertation,  a  characterization  of  GUIDON'S  place  in  the  ICAI  field  in  terms  of 
dimensions  for  comparing  research  interests,  a  review  of  future  research  directions 
for  GUIDON,  and  speculation  about  the  future. 


10.1  Accomplishments 


The  contributions  of  this  dissertation  to  research  in  ICAI  can  be  grouped  into 
four  categories:  method,  discourse,  knowledge  representation,  and  pedagogy. 


METHOD 


I.  Applied  knowledge  engineering  to  development  of  a  tutor: 
Designed  a  framework  for  formalizing  tutorial  dialogue  expertise, 
and  developed  a  corpus  of  rules  using  the  same  approach  taken  for 
developing  knowledge-based  consultation  systems. 


II.  Extended  concept  of  a  problem-solution  trace  to  construct 
a  complete  AND/OR  tree  from  a  rule-based  consultation.  Used  the 
tree  for  guiding  tutorial  dialogue,  testing  the  student's 
understanding,  and  (overlay)  modeling  for  hypothesis  evaluation. 


DISCOURSE 


III.  Analyzed  capabilities  of  MYCIN's  question/answer  system  in 
terms  of  the  basic  requirements  of  an  explanation  system. 


IV.  Applied  and  extended  natural  language  ideas  about 
discourse  to  tutorial  dialogues;  described  the  "dialogue 
management"  problem  In  terms  of  these  constraints: 

—  knowledge  situation  complexity 

—  the  student  model 
--  domain  logic 

—  tutorial  goals 

—  conversational  postulates,  e.g.,  economy 
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V.  Emphasized  importance  of  providing  for  and  coping  with 
student  initiative:  Formalized  discourse  procedures  and  means  for 
sharing  context  that  make  the  tutor  responsive  to  what  the  student 
knows,  wants  to  know  more  about,  or  wants  to  ignore. 


KNOWLEDGE  REPRESENTATION 


VI.  Analyzed  and  characterized  the  levels  of  knowledge  and 
kinds  of  abstractions  contained  in  MYCIN's  rules. 


A.  Other  forms  of  knowledge  useful  for 
understanding  and  remembering  the  performance 
rules:  structure,  support,  and  strategy 


1.  used  rule  schema  for 
grouping  rules  to  simplify 
summaries  of  evidence 


2.  showed  how  "models  of 
sufficient  evidence"  can  be 
used  to  explain  the  rules 


3.  clarified  relation  of 
problem-solving  strategies  to 
object  level  methods  (rules), 
and  illustrated  how  they  can  be 
used  to  provide  assistance  to 
the  student 


B.  Corollary:  Showed  that  the  student  model 
is  superficial  If  it  contains  only  performance  rules 
(overlay  model). 


PEDAGOGY 


VII.  Developed  techniques  for  conveying  the  structure  of  the 
rule  set  using  rule  models,  subgoal  trees,  and  statistics. 
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VIII.  Implemented  an  hypothesis  revision  strategy  for  providing 
assistance  to  the  student 


IX.  Provided  an  unobtrusive  window  on  the  expert  program's 
reasoning  by  indexed  remarks  for  which  details  are  available:  based 
on  strategy  of  challenging  the  student  to  examine  his 
understanding. 


X.  Constrained  dialogue  to  goal-directed  exploration,  but 
provided  the  means  for  changing  the  topic  arbitrarily,  thus 
permitting  flexible  exploration  of  the  expert's  reasoning 


XI.  Defined  opportunistic  tutoring  to  be  a  tutorial  interruption  for 
the  purpose  of  diagnostic  modeling  or  presentation  of  course 
material,  accommodated  and  suggested  by  the  conventional  or 
logical  properties  of  the  dialogue  situation. 


1 0.2  Dimensions  for  Comparing  and  Critiquing  ICAI  Research 


What  do  ICAI  techniques  buy  us?  What  are  the  dimensions  by  which  we 
measure  the  contributions  that  these  programs  make  to  our  understanding  of  effective 
teaching?  We  have  chosen  five  dimensions  for  comparing  and  critiquing  the 
contributions  of  ICAI  research  to  a  theory  of  Instruction: 


1 )  Format  Clarity.  ICAI  research  involves  more  than  writing 
programs  that  can  teach.  Ideally,  the  product  we  aim  for  is  a  body 
of  explicit  teaching  methods  and  strategies,  with  an  understanding 
of  when  and  why  they  are  appropriate. 


2)  Articulateness.  The  program  must  be  capable  of  more  than 
judging  the  correctness  of  the  student's  solution.  It  should  be  able 
to  talk  about  alternative  solution  steps,  strategies,  provide 
metaphorical  models,  and  draw  analogies  to  explain  its  methods. 


3)  Understanding  of  the  Student.  The  program  should  be  able 
to  do  more  than  determine  whether  a  student's  solution  steps  and 
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answer  are  consistent  with  an  expert  approach.  It  should  be  able 
to  Interpret  the  student's  behavior  and  construct  a  description  of 
his  methods  and  strategies. 


4)  Mixed-Initiative  Dialogue.  Dialogue  flexibility  should  allow 
the  program  to  vary  the  level  of  detail  of  the  discussion  from  fine¬ 
grained  causal  arguments  to  summaries  and  overviews  of  material. 
The  program  should  be  sensitive  to  principles  for  controlling 
verbosity  and  persistence  of  focus. 


5)  Task  Difficulty.  The  usefulness  of  ICAI  hinges  in  part  on 
making  contributions  to  areas  that  are  currently  difficult  or  time- 
consuming  to  teach  by  conventional  methods,  such  as  diagnostic 
reasoning  in  medicine  and  electronics.  An  important  reason  for 
developing  ICAI  programs  is  to  make  available  individualized 
experiential  learning,  an  effective  but  labor-intensive  teaching 
method. 

Subsequent  sections  briefly  characterize  GUIDON'S  contributions  along  these 
dimensions,  emphasizing  the  relation  to  other  work  in  the  field. 


10.2.1  Formal  Clarity 

In  frame-oriented  CAI,  branching  logic  implicitly  represented  the  author’s 
teaching  methods.  Even  when  the  same  principles  might  have  applied  in  several 
situations,  the  designer  had  to  explicitly  specify  which  material  should  be  presented 
in  each  frame,  and  what  transitions  should  be  made  based  on  student  response.  In 
designing  a  tutorial  program  that  separates  domain  knowledge  from  teaching 
expertise,  we  are  forced  to  make  explicit  the  teaching  strategies  that  had  been 
"hard-wired"  In  frame  systems  and  earlier  drill  and  practice  programs. 

The  advantages  of  this  separation  include;  (1)  multiple  uses  of  the  teaching 
strategies,  l.e.,  apply  them  to  multiple  problem  situations  and  perhaps  multiple  domains. 
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(2)  the  ability  to  directly  communicate  the  strategies  to  other  researchers,  (3)  the 
possibility  of  experimenting  with  methods  without  altering  subject  material,  (4) 
modularity  that  enhances  debugging  of  the  program,  e.g.,  in  GUIDON  it  is  especially 
easy  to  determine  which  tutoring  rule  is  responsible  for  a  tutorial  action,  and  ail 
changes  are  local  to  the  discourse  situation. 

It  was  natural  to  represent  GUIDON'S  teaching  expertise  as  a  separate  body  of 
knowledge  because  we  wished  to  leave  the  pre-existing  rule  base  of  MYCIN  intact, 
and  we  wanted  to  experiment  with  each  of  the  consultation  systems  that  used 
MYCIN's  representation.  We  chose  to  represent  GUIDON  as  a  set  of  explicit  rules  that 
could  be  translated  directly  to  English  form  because  of  the  advantage  of  this 
discipline  for  organizing  and  clarifying  knowledge.  Indeed,  we  observe  that  it  is 
common  today  to  list  knowledge  for  discourse  and  teaching  in  the  form  of  production 
rules  [Collins,  1976]  [Carbonell,  1978]  [Burton,  1979].  GUIDON  is  the  first  program 
that  we  know  of  that  internally  represents  tutorial  dialogue  knowledge  in  the  form  of 
production  rules. 

10.2.2  Articulateness  and  Understanding  the  Student 

One  advantage  of  frame-oriented  CAI  is  that  it  could  guide  presentation  on  a 
fine-grained  level.  This  degree  of  control,  which  tied  very  specific  responses  to 
particular  student  behavior,  was  achieved  by  providing  the  program  with  fixed-text 
for  many  possible  situations.  Some  of  the  most  sophisticated  systems  achieved  a 
very  human-like  interaction  by  conditionally  choosing  remarks  on  the  basis  of  the 
history  of  the  dialogue  [Feurzalg,  at  al.,  1964].  However,  these  systems  were 
useless  when  the  student  behaved  in  a  way  that  was  not  anticipated  by  the  author. 
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It  turns  out  that  a  human  tutor  (like  the  frame  author)  draws  from  many 
resources  when  detecting  student  errors  and  presenting  new  information.  An 
important  dimension  for  comparing  ICAl  systems  Is  the  kind  of  domain  knowledge 
accessible  to  the  system  and  what  it  allows  the  tutor  to  do.  The  following  list  is  a 
progressive  characterization  of  the  capabilities  of  ICAl  systems: 

1)  judge  overall  correctness  of  an  answer  (SCHOLAR,  BIP) 

2)  judge  consistency  of  an  answer  with  respect  to  known  data 
(SOPHIE) 

3)  articulate  "issues"  or  considerations  that  a  student  (or 
expert)  solution  exhibits  (WEST) 

4)  articulate  an  understanding  of  student  behavior  in  terms  of 
a  chain  of  concepts  and  reasoning  steps  that  an  expert  might  use 
(WUMPUS,  GUIDON) 

5)  articulate  methods  and  strategies  that  vary  from  expert 
performance  (BUGGY,  WHY) 

It  is  implicit  in  this  list  that  capabilities  are  cumulative.  For  example,  WHY  can 
(or  could,  if  programmed  to  do  so)  explain  a  deduction  in  terms  of  specific  reasoning 
steps,  and  SOPHIE  can  judge  the  correctness  of  an  answer,  but  not  articulate  the 
reasoning  that  makes  it  right  In  causal  terms.1  GUIDON'S  capabilities  derive  directly 
from  the  production  system  representation  of  MYCIN  that  makes  It  possible  to 
construct  a  deep  structure  interpretation  of  a  problem.  Evidence  is  tied  to  the 


1  SOPHIE  could  give  model-theoretic  (logic)  arguments.  For  example,  to  explain 
why  two  measurements  are  redundant,  it  could  say  that  the  second  measurement 
does  not  alter  the  space  of  hypotheses  that  are  possible.  However,  this  kind  of 
explanation  is  not  very  useful  to  a  student. 
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solution  In  terms  of  intermediate  concepts  and  problem-solving  methods  (the  rules). 
This  level  of  representation  gives  the  potential  for  providing  "best  next  step" 
assistance  to  the  student,  as  well  as  detecting  lines  of  reasoning  he  may  be  ignoring. 
GUIDON'S  student  model  makes  a  distinction  between  knowledge  of  a  problem-solving 
method  and  its  actual  application  for  the  given  task.  This  distinction  becomes 
important  within  the  dynamics  of  the  dialogue  in  which  new  knowledge  about  the 
problem's  solution  Is  gradually  revealed  to  the  student. 

However,  GUIDON'S  diagnostic  capabilities  are  limited  by  its  inability  to  construct 
a  plausible  interpretation  of  non-expert  reasoning,  as  BUGGY  can.  And  GUIDON  cannot 
use  multiple  models  for  understanding  causal  rules,  as  has  been  proposed  for  the  WHY 
system.  It  should  be  understood  that  these  are  not  necessarily  crippling  weaknesses, 
so  much  as  simplifications  that  must  be  kept  in  mind  as  research  when  evaluating  the 
tutorial  strategies  and  dialogue  capabilities  described  in  this  dissertation. 

10.2.3  Mixed-Initiative  Dialogue  and  Task  Difficulty 

A  good  author  of  a  frame -oriented  CAI  program  could  organize  and  individualize 
lesson  material  so  that  the  course  would  engage  the  student  and  keep  pace  with  his 
understanding.  Just  as  subject  material  wasn't  separated  from  teaching  strategy,  no 
distinction  was  made  between  principles  of  discourse  and  pedagogical  methods.  ICAi 
research  has  just  begun  to  formalize  the  principles  of  a  motivating,  easy-to-follow 
style  of  presentation.  A  major  contribution  of  GUIDON  is  its  framework  of  tutoring  rules 
which  make  the  program  sensitive  to  nuances  of  domain  structure,  student 
differences,  and  dialogue  conventions. 
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10.2.3.1  Dialogue  in  ICAI  programs 

While  one  distinguishing  feature  of  ICAI  programs  is  frequently  cited  to  be  their 
capability  to  engage  the  student  In  a  mixed-initiative  dialogue  [Hart.  1075]  [Hart  & 
Koffman,  1076],  we  find  that  the  "natural  language  understanding"  in  these  programs 
involves  parsing  student  input  [Burton,  1076]  and  not  conversational  Interaction. 

In  the  reactive  environment  of  the  early  SOPHIE  lab  [Brown,  Rubinstein,  &  Burton, 

1076]  the  tutor  never  takes  the  initiative  at  all,  and  in  the  games  of  WEST  [Burton  & 
Brown,  1076]  and  WUMPUS  [Goldstein,  1077],  the  tutor's  remarks  are  all 
Interruptions  or  reactions  enabled  by  the  immediately  preceding  move  taken  by  the 

student.1  SCHOLAR  [Cerbonell,  1070]  [Collins,  1076]  and  WHY  [Stevens  &  Collins, 

1077]  [Stevens,  Collins,  &  Goldin,  1076],  the  geography  and  meteorology  tutors, 
follow  a  Socratic  dialogue  format  of  repetitive  questioning  using  topic  selection  rules 
and  strategies  for  testing  a  student's  understanding;  the  session  itself  involves  no 
overarching  problem  or  shared  task  to  be  discussed.  ABLOCKS  [Brown,  et  al.,  1 076] 
follows  similar  initiative  strategies  for  getting  the  student  to  understand  that  a 
question  Is  redundant,  or  that  a  deduction  can  be  made,  but  again  the  sequence  of 
the  dialogue  interaction  is  short  and  focused  on  one  reasoning  step.  Thus,  ICAI 
research  has  emphasized  the  use  of  domain  expertise  for  modeling  the  student,  and 
tutoring  principles  for  correcting  his  misconceptions,  but  it  has  dealt  only  tangentially 
with  the  problems  of  carrying  on  a  prolonged,  purposeful  tutorial  dialogue. 

Oiscourse  knowledge  Is  represented  explicitly  in  GUIDON,  using  some  of  the 


1  Of  course  the  evolving  model  of  the  student  and  history  of  past  moves  is  an 
important  and  non-trhrial  reason  for  making  an  Interruption. 
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same  techniques  for  formalizing  expertise  that  were  developed  in  Al  problem-solvers 
like  MYCIN.  Tutoring  rules  select  alternative  dialogues  on  the  basis  of  principles  of 
economy,  domain  logic,  and  tutoring  or  modeling  goals.  Arranged  into  procedures, 
these  rules  cope  with  various  recurrent  situations  in  the  tutorial  dialogue— e.g., 
examining  a  student's  understanding  after  he  asks  a  question  that  shows  unexpected 
expertise,  relating  an  inference  to  one  Just  discussed,  and  giving  advice  to  the 
student  after  he  makes  an  hypothesis  about  a  subproblem. 

10.2.3,2  Relation  to  Natural  Language  Research 

The  discourse  problems  dealt  with  here  include  maintaining  and  sharing  context 
as  solution  of  the  task  proceeds  and  providing  means  for  the  student  to  express 
initiative  as  he  unfolds  the  complexity  of  the  problem  and  encounters  limitations  in  his 
understanding.  Other  researchers  have  reported  various  theoretical  aspects  of  task- 
oriented  dialogues,  such  as  recognizing  and  generating  intentions  and  plans  [Faught, 
1977]  [Carboneli,  1978]  and  focusing  on  objects  and  subtasks  [Grosz,  1977].  We 
distinguish  our  work  from  other  natural  language  research  In  two  ways.  First,  GUIDON 
does  not  incorporate  a  comprehensive  model  of  discourse  like  GUS  [Bobrow,  et  al., 
1977].  Second,  we  are  formalizing  tutorial  knowledge  In  terms  of  specific 
performance  rules  for  various  situations,  rather  than  studying  human  dialogues  and 
enumerating  general  dialogue  interaction  principles. 

We  view  the  discourse  problem  as  requiring  (a)  interpretation  of  intent  and  (b) 
reasoning  about  this  understanding  in  order  to  act.  We  have  minimized  the 
understanding  problem  by  restricting  the  student  to  a  command-oriented  input,  thus 
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avoiding  the  problem  of  recognizing  new  dialogue  situations  in  free  text.  This  has 
allowed  us  to  concentrate  on  the  tutorial  knowledge  for  generating  remarks  and 
guiding  the  dialogue.  Consequently,  we  have  not  dealt  In  much  generality  with  many 
theoretical  issues  of  task-oriented  dialogues,  e.g.,  the  problem  of  focus  in  the  SRI 
assembly  task  [Oeutsch,  1974]  [Grosz,  1977].  Generation  of  remarks  is  on  the  level 
of  choosing  the  course  of  the  dialogue,  rather  than  building  up  output  from  grammatical 
components.  We  devote  just  a  small  amount  of  computation  towards  determining  the 
student's  Intentions,  chiefly  by  relating  student  requests  for  data  to  the  current 

topic.1  Moreover,  the  AND/OR  tree  of  topics  and  rules  strongly  constrains  the  tutor's 
choice  of  dialogue  situations:  in  less  rigidly  pedantic  dialogues,  generation  of 
conversation  is  sensitive  to  considerations  such  as  the  social  context  of  the 
participants  and  emotional  connotation  [Winograd,  1977].  (Some  of  Burton  and 
Brown's  kibitzing  strategies  are  like  this  [Burton,  1979].) 

1 0.3  Contribution  to  Medical  CAI 

GUIDON  Is  the  first  medical  tutorial  program  that  we  know  of  that  uses  ICAl 
techniques.  Relevant  contributions  to  medical  CAI  Include:  greater  individualization  of 
tutorials,  a  framework  for  expressing  end  accumulating  tutorial  dialogue  expertise,  and 
a  language  for  diagnostic  problem-solving  strategies.  By  constructing  a  model  of 
problem-solving  strategies  In  a  student  model,  something  traditional  medical  CAI 
systems  do  not  even  consider,  ICAl  systems  could  provide  a  basis  for  critiquing  and 
teaching  diagnosis  in  terms  that  even  go  beyond  classroom  or  clinical  experience. 

1  However,  we  do  intend  to  build  a  more  complex  model  of  the  student's 
strategies,  and  this  might  involve  parsing  complex  input  (as  in  Sleeman's  program  for 
understanding  student  explanations  [Slesman,  1977]). 
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10.4  Future  Directions  for  ths  GUIDON  System 

Initially,  we  will  experiment  with  and  make  straightforward  additions  to  the 
existing  GUIDON  program.  For  example,  26X  of  MYCIN'a  rules  have  iterations  in  the 
premise,  yet  there  are  no  methods  for  discussing  these  rules  except  clause-by¬ 
clause.  It  should  be  possible  to  find  patterns  in  these  rules  and  design  concise 
methods  for  presenting  them.  Moreover,  there  are  a  host  of  subtle  t-rules  that  were 
left  out  of  the  Initial  system  because  of  the  difficulty  of  implementing  seemed  to 
outweigh  the  advantages  of  their  infrequent  application.  For  example,  a  rule  that  fails 
to  apply  for  the  sole  reason  that  case  data  are  missing  is  interesting  if  the  conclusion 
made  by  the  rule  would  significantly  alter  the  outcome  of  the  case.  Unfortunately, 
detection  of  this  situation  involves  a  fairly  complex  analysis  of  the  hypothetical  case, 
essentially  requiring  construction  of  a  parallel  AND/OR  tree  that  simulates  the  changed 
conclusions.1 

Aside  from  practical  extensions  of  current  techniques,  there  are  basic 
theoretical  problems  that  remain  to  be  worked  out.  Many  of  these  issues  of 
discourse,  tutoring,  modeling,  knowledge  representation,  and  learning  have  been 
described  at  some  length  in  the  body  of  this  dissertation.  These  issues,  with  some 
new  twists  we  have  not  considered  previously,  are  summarized  below. 


1  The  same  facility  could  be  used  to  respond  to  student  questions  like,  "Would 
you  still  treat  for  E.coll  If  the  patient  were  younger  than  12?"  This  form  of 
hypothetical  inquiry  was  made  possible  In  SOPHIE  by  simulation  of  the  electronic 
circuit. 
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10.4.1  Dialogue  Management 

GUIDON'S  guidance  of  the  dialogue  is  limited  to  one  level  of  the  solution:  there 
are  methods  to  discuss  a  goal  and  methods  to  discuss  a  rule.  Analysis  of  the  SACON 
dialogue  shows  that  dialogue  management  should  be  more  comprehensive.  There  are 
at  least  two  reasons  for  this.  First  Is  the  need  for  overviews:  we  saw  that  the  SACON 
dialogue  was  tedious  when  GUIDON  methodically  guided  the  dialogue  In  a  depth  first 
way,  only  motivating  the  discussion  one  step  at  a  time.  GUIDON  cannot  convey  a 
unified  sense  of  purpose  because  it  has  no  idea  which  topics  are  more  important  than 
others. 

Second,  the  constraint  on  time  for  the  session  prevents  treating  each  "context" 
(e.g.,  culture,  loading,  organism)  with  equal  emphasis;  some  subproblems  (parameters) 
'must  be  skipped  over  or  mentioned  In  passing.  While  GUIDON  has  a  number  of  methods 
for  reducing  the  number  of  rules  that  are  mentioned  and  has  methods  for  mentioning 
them  economically,  it  never  looks  down  into  the  solution,  beyond  one  level  of  the  tree, 
to  decide  what  topics  it  should  really  be  spending  time  on. 

It  is  clear  that  more  research  remains  to  be  done  on  the  subject  of  bow  people 
focus  discussion  of  a  long,  complicated  problem.  The  program  will  need  discourse 
methods  for  partial  presentation  that  allow  it  to  call  subproblems  to  the  student's 
attention  without  "discussing"  them  In  the  kind  of  formal  procedure  that  GUIDON 
follows.  This  might  just  be  practical  extension  along  the  lines  of  methods  we  have 
already  implemented  for  special  goals  (no  rules  succeed,  one  definitional  rule  gives 
the  answer).  On  the  other  hand,  more  domain  knowledge  will  be  required  for  the 
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program  to  know  what  Is  special  or  essential  about  the  case  being  presented  to  the 
student.  Given  that  the  solution  is  already  worked  out  in  the  AND/OR  tree,  a 
discussion  might  be  planned  ahead  of  time.  Dialogue  planning  would  probably  relate 
directly  to  strategies  for  case  selection,  so  that  topics  could  be  omitted  in  one  case 
on  the  basis  that  they  would  be  raised  at  another  time. 

A  rudimentary  lesson  plan  would  be  a  list  of  topics  and  rules  that  GUIDON  would 
attempt  to  discuss  during  the  case  session.  This  plan  would  control  opportunistic 
tutoring  strategies  and  methods  for  discussing  goals  and  rules.  Ideally,  lesson 
planning  would  incorporate  causal  models  and  strategies  for  applying  rules  (doing 
diagnosis),  so  that  discussion  of  material  would  be  progressive  and  coherent.  Thus, 
the  program  would  quiz  about  a  rule  not  Just  on  the  basis  of  whether  the  student  knew 
it  or  not,  or  its  "sophistication."  instead  quizzes  would  be  motivated  by  an  underlying 
model  of  the  domain  or  diagnostic  strategy  that  the  program  was  really  trying  to 
teach. 

Case  selection  strategies  might  be  based  simply  on  statistical  properties  of  the 
library  of  cases.  A  core  set  of  cases  might  be  chosen  to  cover  the  rule  set  at  a 
certain  level  of  sophistication.  However,  if  we  are  trying  to  teach  more  than  individual 
rules,  then  even  if  the  statistical  analysis  captures  a  good  set  of  examples,  the 
program  will  not  be  able  to  articulate  the  principles  that  make  the  cases  worth 
studying.  For  example,  if  we  wish  to  convey  underlying  models  of  disease,  then  some 

e 

topics  will  have  intrinsic  importance:  they  will  be  essential  for  interpreting  the  solution 
of  the  case  in  terms  of  an  analogy  with  the  mode).  Moreover,  if  we  wish  to  teach 
prototypical  information,  like  a  typical  case  of  meningitis  in  a  child,  we  will  find  it 
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useful  to  have  information  about  normal  and  expected  ranges  of  case  data.  In  this 
case,  some  topics  will  have  extrinsic  importance:  they  will  be  interesting  findings  in 
relation  to  the  range  of  cases  we  might  expect  to  encounter.  (Information  that  is 
rarely  available  Is  probably  worth  mentioning  when  it  is  known,  no  matter  how 
insignificant  the  evidence  It  contributes.)  Given  these  distinctions  about  types  of 
cases,  case  selection  rules  would  be  able  used  to  choose  "training  examples"  for 
particular  students. 

Finally,  getting  back  to  the  more  local  kind  of  dialogue  problems  that  have  been 
the  focus  of  WHY  and  A0LOCKS  research,  we  note  that  real  give  and  take  is  not 
possible  In  GUIDON  dialogues.  Instead  of  pursuing  the  student's  reasoning,  the 
program  corrects  him  immediately.  Using  the  existing  framework,  we  should  consider 
incorporating  tutoring  transitions  that  track  down  a  misconception,  similar  to  those 
used  in  WHY.  Some  of  the  problems  of  doing  this  were  mentioned  in  Section  0.2.3.2  in 
the  context  of  the  hypothesis  response  part  of  the  dialogue. 

1 0.4.2  Tutoring 

The  current  version  of  GUIDON  is  best  considered  as  a  default  collection  of 
tutoring  rules  that  have  been  culled  from  the  literature  and  intuition.  Research  will 
proceed  by  conducting  Informal  experiments  at  first  with  students  having  different 
levels  of  expertise.  Examination  of  protocols  will  lead  to  tuning  and  extension  of  the 
current  set  of  200  tutoring  rules.  From  this  experience,  formal  experiments  for 
testing  specific  strategies  might  then  be  carried  out.  One  experiment  might  try  to 
determine  whether  continued  development  of  the  system  should  emphasize  passive 
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devices  by  which  the  student  can  discover  new  connections  at  his  own  pace  or 
whether  the  program  should  take  a  very  active  role  of  presenting  new  rules  and 
strategies. 

Before  testing  the  effectiveness  of  GUIDON'S  tutoring  methods  further,  we  will 
want  to  make  explicit  the  discourse  principles  that  many  t-rules  have  in  common,  in 
particular,  the  kind  of  constraints  that  justify  transitions  between  discourse  situations 
should  be  made  explicit.  For  example,  it  appears  that  the  dialogue  excerpt  of  Figure 
6.13  is  disconnected  because  the  tutor  did  not  follow  the  principle  of  logically 
considering  all  aspects  of  the  situation  before  offering  the  student  possible  ways  of 
proceeding. 

10.4.3  Modeling 

Development  of  the  current  student  model  has  focused  on  evaluating  a  student 
hypothesis  for  a  subproblem,  it  would  be  Interesting  to  investigate  the  other  portion 
of  the  dialogue— gathering  data.  Why  does  the  student  ask  a  particular  question? 
Can  we  silently  follow  his  approach,  understanding  it  to  be  a  coherent  application  of 
domain  rules?  Research  into  diagnostic  question-asking  should  be  relevant  here  (e.g.. 
[Sprosty,  1963]  and  [Elstein,  Shulman,  &  Sprafka,  1978]).  If  we  determine  that 
the  student  is  gathering  data  in  blocks  (e.g.,  laboratory  data  first),  or  that  he  is 
working  on  a  given  hypothesis,  we  will  be  In  a  better  position  to  offer  relevant  help 
when  he  requests  it.  The  sample  protocol  suggested  good  leads  for  this  research, 
including  a  model  of  cognitive  style,  or  approach  to  the  problem  (see  Section  3.4.2). 
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An  accurate  model  of  why  the  student  is  asking  a  question  would  be  particularly 
useful  if  we  were  to  make  the  dialogue  frte-form,  instead  of  maintaining  goal- 
directedness.  An  "irrelevant  question"  would  then  be  an  inquiry  that  the  program 
could  not  fit  into  its  model  of  the  student's  strategy.  Certainly  the  tutor  could 
interrupt  like  a  human  tutor  would  when  unable  to  follow  the  student's  reasoning. 
Following  the  student  might  enable  the  tutor  to  provide  much  better  assistance,  for 
suggestions  could  reflect  the  approach,  as  well  as  the  work  that  had  been  completed 
so  far. 

A  completely  different  improvement  to  the  overlay  model  would  be  to  incorporate 
a  critic  that  would  provide  feedback  about  the  accuracy  of  the  modeling  process.  The 
overlay  model  is  updated  on  a  rule-by-rule  basis  that  leads  the  program  to  conclude 
that  the  student  probably  knows  evidence  based  on  a  chain  of  reasoning  steps.  If 
the  student's  hypothesis  does  not  agree  with  this  prediction,  then  the  tutor  might 
reason  backwards  to  determine  the  rules  and  subgoals  that  the  student  did  not 
consider.  The  critic  would  then  step  in  to  reexamine  the  tutoring  rules  that  were  used 
to  construct  the  student  model,  for  these  rules  are  now  suspect.  Perhaps  parameters 
could  be  altered  that  would  tune  the  modeling  rules  to  the  individual  student.  More 
speculatively,  we  might  imagine  that  the  program  could  learn  better  modeling  rules  as 
a  result  of  Its  interactions  with  many  students.  (See  [Carr  &  Goldstein,  1977]  and 
[Burton,  1979]  for  a  related  discussion  of  adaptive  modeling.) 

Finally,  we  have  not  discussed  the  problem  of  keeping  records  of  tutorial 
sessions.  CAI  provides  a  unique  opportunity  for  compiling  data  about  student  behavior 
that  can  be  used  to  simulate  students  and  experiment  with  different  modeling  and 
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teaching  strategies  [Wescourt,  at  at..  1976].  The  current  version  of  the  program 
maintains  two  typescript  files,  one  that  is  an  exact  copy  of  what  appears  on  the 
student's  terminal,  and  another  that  indicates  internal  states  of  the  program.  The 
most  relevant  data  to  be  collected  are;  1)  student  use  of  options  including  requests 
for  data  and  assistance;  2)  modeling  decisions  made  by  the  program;  3)  strategical 
decisions  made  by  the  program  (application  of  t-rules)  and  4)  discourse  situations 
encountered  during  the  dialogue.  A  practical  extension  to  the  current  system  would 
be  to  make  the  internal  trace  succinct  and  machine-readable  so  that  it  could  be  used 
in  simulation  studies. 

1 0.4.4  Knowledge  Representation 

Chapter  5  described  forms  of  knowledge  that  might  be  added  to  the  program: 
structure,  support,  and  strategy.  In  the  discussion  of  automatic  case  selection,  we 
pointed  out  that  prototypical  examples  would  also  be  useful.  Considerable  theoretical 
research  remains  to  be  done  to  decide  how  this  materia)  should  be  represented  and 
acquired. 

It  would  be  interesting  to  study  the  applicability  of  GUIDON'S  discourse  methods 
to  non-rule-based  representations.  For  example,  suppose  that  the  representation  of 
knowledge  consisted  of  relations  that  were  inference  triggers  (e.g.,  as  in  INTERNIST 
[Pople,  1977]),  causal  links  (e.g.,  as  in  CASNET  [Weiss,  Kulikowski,  &  Safir,  1977]), 
or  even  hierarchical  mechanistic  models  [Brown,  1977a],  we  still  might  find  it 
convenient  to  think  of  the  dialogue  as  being  structured  into  goals  (something  to  find 
out),  case  data,  possible  outcomes  (alternative  hypotheses),  and  inference  relations. 


Future  Directions  for  the  GUIDON  System 


357 


It  seems  probable  that  many  of  the  issues  of  dialogue  management  we  have  discussed 
will  be  important,  though  we  may  need  to  augment  the  situations  we  have  formalized 
(Figure  6.3).  For  example,  for  an  hypothesis-oriented  discussion  we  would  refine  the 
GOAL  procedure  to  focus  on  one  outcome  value  at  a  time,  and  incorporate  transition  t- 
rules  for  detecting  and  discussing  shifts  to  another  outcome  (hypothesis  revision). 

10.4.5  Learning 

Few  ICAI  systems  include  a  learning  module  [Goldstein,  1877]  [O'Shen  > 
to  serve  as  a  critic  of  the  tutoring  methods  used  by  the  program.  The  problem  is  to 
assign  credit  and  blame  to  the  tutoring  methods  on  the  basis  of  evidence  that  they 
succeeded  or  failed  in  teaching  material  to  the  student.  The  system  then  adapts  Its 
tutoring  methods  to  improve  itself,  as  we  described  in  the  case  of  adaptive  modeling. 
However,  It  is  very  difficult  to  separate  effectiveness  of  tutoring  methods  from  the 
noise  contributed  by  student  forgetting  and  the  fact  that  certain  material  might  be 
mentioned  in  different  ways  on  several  occasions  In  different  problems. 

10.5  Speculation  about  the  Future:  Cognitive  Science 

Intelligent  Computer  Aided  Instruction  Is  one  subfield  of  At  that  has  become 
Infused  with  ideas  about  human  cognition.  In  the  late  1970s  we  observe  a  shift  in  the 
kinds  of  questions  researchers  are  asking.  Collins  is  asking  not  just  how  to  make  a 
program  answer  a  student's  questions,  but  "What  is  a  humanly  plausible  way  of 
reasoning  with  uncertain  knowledge?"  Burton  and  Brown  are  asking  more  than  whether 
a  student  knows  a  game  rule,  but  "What  kinds  of  assumptions  might  he  be  making 
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about  the  purpose  of  the  game?"  Goldberg  wanted  to  do  more  than  correct  a 
student's  proof,  but  also  complete  a  partial  solution,  and  asked,  "What  kinds  of  proofs 
do  students  write?"  Stevens  goes  beyond  telling  the  student  that  a  causal  argument 
is  insufficient,  he  asks,  "What  model  of  the  world  would  suggest  this  incorrect 
prediction?"  Similarly,  in  examining  justifications  physicians  gave  for  MYCIN's  rules, 
we  asked,  "What  makes  an  explanation  of  these  rules  satisfying?"  These  are  some  of 
the  questions  of  cognitive  science,  a  study  rooted  in  the  powers,  limitations  and 
habits  of  human  thinking. 

In  a  way,  it  is  not  too  surprising  that  researchers  attempting  to  design  programs 
that  can  teach  have  turned  to  a  study  of  how  people  think.  We  are  starting  to  realize 
that  there  are  constraints  that  affect  how  people  make  sense  of  new  information  and 
how  they  explain  it  to  others.  We  don't  know  very  much  about  these  constraints 
today.  We  see  evidence  of  memory  limitations  and  a  tenoency  to  associate 
knowledge  in  certain  ways,  but  we  don't  know  much  more  than  that.  Maybe  the 
evolving  studies  that  characterize  what  people  know,  what  they  really  remember 
about  what  they  claim  to  know,  and  how  they  use  this  knowledge  to  solve  problems 
will  turn  out  to  more  useful  to  teaching  than  these  basic  "hardware"  limitations. 

The  title  of  this  dissertation  mentions  the  transfer  of  expertise.  The  very  fact  that 
we  are  attempting  to  transfer  knowledge  through  an  interactive  process  between  man 
and  machine  makes  the  study  of  human  thinking  important.  Indeed,  after  several 
years  of  acquiring  expertise  from  human  experts  and  formalizing  it  for  our  programs, 
we  have  learned  that  we  are  capturing  but  a  small  part  of  what  these  people  know. 
Moreover,  we  have  witnessed  tne  laborious  process  of  writing  down  these  principles, 
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and  realized  that  the  real  heart  of  our  problem  might  be  to  understand  better  how 
experts  prefer  (or  need  to)  talk  about  what  they  know.  One  physician  hands  ur  a 
flowchart  and  we  say,  "Fine,  now  convert  them  to  rules."  An  engineer  is  told,  "Write 
rules!"  and  in  a  day  has  200  of  them,  but  still  can't  explain  to  the  computer  scientist 
how  he  solves  a  problem.  It  seems  obvious  that  ICAI  will  benefit  from  the  study  of 
how  people,  who  already  know  what  we  want  to  teach,  organize  and  make  use  of  their 
knowledge.  There  may  be  many  moltings  in  between,  but  at  least  we  will  know  what 
we  are  aiming  for. 

In  fact,  the  symmetry  of  knowledge  acquisition  from  a  human  expert  and 
computer-aided  instruction  suggests  that  some  day  these  program-,  may  [Barr, 

Bennett,  &  Clancey,  1979].  By  integrating  the  tutor  with  the  knowledge-acquisition 
program,  we  would  be  in  a  better  position  to  teach  the  material  later,  particularly  if  the 
way  that  the  expert  talks  about  what  he  knows  reflects  an  underlying  set  of  human 
constraints  that  is  shared  by  the  student  who  wants  to  make  sense  of  the  same 
material.  We  envision  a  tutor  that  would  ask  questions  of  the  domain  expert  that 
would  be  relevant  for  teaching  the  knowledge  it  is  acquiring.  To  do  this,  the  tutor 
would  need  to  have  a  good  model  of  the  constraints  that  will  guide  its  interaction  with 
the  student,  that  is,  a  good  model  of  how  people  learn.  As  matter  of  fact,  the  tutorial 
program  would  be  simulating  a  student  as  it  posed  these  questions  and  fit  new  facts 
into  its  knowledge  base.  Rather  than  reading  a  rule's  premise  and  conclusion  and 
hooking  it  into  a  list  for  later  retrieval,  knowledge  acquisition  by  a  program  would  have  to 
be  something  closer  to  human  learning. 


Finally,  the  principles  of  discourse  that  now  constitute  GUIDON'S  tutorial 
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expertise  could  be  thought  of  as  being  Inverted  as  knowledge  is  acquired  by  the 
system.  Once  we  manage  to  capture  the  expertise  that  good  teachers  use,  then 
what  GUIDON  would  say  in  a  certain  situation  if  it  were  teaching  a  student  should  be 
analogous  to  what  It  hears  when  a  human  expert  Is  explaining  something  to  it.  We 
might  imagine  a  single  corpus  of  knowledge  that  characterized  how  people  learn  and 
explain  what  they  know,  and  this  transftr  of  expertise  expertise  could  be  used  both  to 
drive  the  interaction  with  a  human  expert,  making  provision  for  later  teaching  to  a 
student,  and  to  drive  the  tutorial  Interaction  with  a  student.  These  principles  might  be 
the  foundation  for  a  science  of  cognition. 
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Appendix  A 

Formation  of  the  AND/OR  Solution  Tree 


The  AND/OR  tree  used  by  GUIDON  is  a  reconfiguration  of  the  output  of  a  MYCIN 
consultation.  This  reconfiguration  requires  a  non-trivial  analysis  of  the  rules  and 
conclusions  made  by  MYCIN.  The  analysis  occurs  on  three  recurring  levels:  a  goal,  the 
rules  that  are  used  to  determine  it,  and  the  subgoals  that  are  needed  to  evaluate  a 
rule. 


Figure  A.1.  Three  Levels  of  Analysis  for  Constructing  the  AND/OR  Tree 


The  difficult  part  of  this  analysis  is  on  the  third  level— forming  a  logical 
representation  of  the  rule  in  terms  of  subgoals  using  only  the  functions  AND  and  OR. 
For  example,  the  following  Is  a  rule  and  the  representation  of  why  it  failed  in  a 
particular  consultation: 

RULE517 


If:  1)  The  Infection  which  requires  therapy  Is  meningitis, 

2)  A:  A  smear  of  the  culture  was  not  examined,  or 

B:  Organisms  were  not  seen  on  the  stain  of  the  culture 

3)  The  type  of  the  infection  Is  bacterial,  and 

4)  The  age  of  the  patient  is  less  than  9  years, 

5)  There  Is  evidence  of  congenital  malformation  involving  the  central 
nervous  system, 

Then:  There  Is  evidence  that  the  organisms  (other  than  those  seen  on 
cultures  or  smears)  which  might  be  causing  the  infection  is 
e.coll  (.75),  staphylococcus-coag-pos  (.75),  klebsiella- 
pneumonlae  (.75)  and  pseudomonas-aeruginosa  (.5) 

FAIL-PARM5! :  (OR  (4  (AGE  .  PATIENT-538 ))( 5  (CNSMAIFORM  .  PATIENT-538))) 

Notice  that  goals  are  indicated  by  a  pair  of  the  form  (<parameter>  .  <context>).  The 
entire  AND/OR  tree  consists  of  pointers  between  parameter/context  pairs  and 
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rule/context  pairs.1  This  representation  enables  GUIDON  to  determine  efficiently 
whether  or  not  a  rule  "fires"  without  re-evaluating  the  preconditions  of  the  rule.  For 
example,  the  rule  above  will  fire  as  soon  as  the  age  of  the  patient  or  whether  he  had 
a  congenital  malformation  Is  known.  (Section  8  describes  the  overlay  model  in  detail.) 


The  difficulty  of  the  analysis  stems  from  the  variety  of  predicate  functions  that 
appear  in  MYCIN  rules.  Figure  A.2  classifies  these  functions  into  categories  that  we 
will  refer  to  frequently  during  the  discussion  that  follows. 


Figure  A.2.  Classification  of  MYCIN  Predicate  Functions 


There  are  three  basic  kinds  of  functions: 


1 )  SIMPLE  functions  such  as  SAME  and  MIGHTBE  generally 
Involve  one  subgoal,  e.g.,  "the  site  of  the  culture  is  blood" 
references  the  subgoal  SITE. 


2)  COMPOUND  functions  such  as  SAND  are  composed  of  simple 
functions. 


1  MYCIN  parameters,  such  as  SITE  ("the  site  of  the  culture"),  are  said  to  be 
traced  in  particular  contexts.  Similarly,  rules  are  applied  in  specific  contexts.  Thus, 
each  node  of  the  AND/OR  tree  is  really  a  dotted  pair. 
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3)  COMPLEX  functions  perform  arbitrary  computation,  generally 
involving  several  subgoals.  For  example,  one  of  the  two  complex 
functions  In  MYCIN  Is  described,  "Count  the  number  of  cultures  from 
the  same  site  which  were  taken  since  this  time,  and  .  .  .  find  the 
most  recent  of  these  cultures  which  was  positive  for  an  organism 
with  the  same  identity  as  this  organism...." 


The  input  to  the  program  that  builds  the  AND/OR  tree  consists  of  the  MYCIN 
knowledge  base  of  rules  and  parameters  and  the  result  of  a  particular  consultation.  It 
was  necessary  to  augment  the  consultation  record  (originally  designed  for  use  by  the 
post-consultation  question/answer  program)  to  indicate  In  more  detail  which  rule 
clauses  were  evaluated  and  whether  or  not  they  were  found  to  be  true.  Overall,  the 
analysis  is  a  two-step  process: 


Evaluate  rule,  recording 

Examine  functions  to 

context  of  evaluation 

- \ 

determine  which 

and  truth  value  of  each 

) 

subgoals  are  required 

function 

traces 

&  build  AND/OR  tree 

During  Consultation  Analysis  in  batch  mode 


Figure  A.3.  Two-step  Process  for  Constructing  the  Tree 


It  would  be  possible  for  the  construction  of  the  AND/OR  tree  to  proceed  during 
the  consultation  by  modifying  the  predicate  functions  themselves.  However,  to  keep 
the  tutorial  package  independent  of  the  consultation  system,  and  to  prevent  slowing 
down  the  consultation.  It  was  deemed  better  to  separate  the  analysis  Into  two  steps. 


The  consultation  record  consists  of  rule  conclusions,  final  values  for  goals  that 
were  pursued,  and  a  trace  of  the  evaluation  of  each  rule  (see  Figure  4.6).  This  trace 
is  stored  as  a  property  of  the  rule.  It  Is  built  recursively  by  the  predicate  functions 
as  each  clause  of  the  rule  is  considered.  The  format  of  the  trace  is  shown  in  Figure 

A. 4  In  Backus-Normal  form.1  An  APPLTRACE  is  stored  as  the  APPLIED-TO  property  of  a 
rule. 


In  the  simplest  form,  a  trace  consists  of  the  context  in  which  the  rule  was 
evaluated  and  the  fate  of  the  evaluation:  NIL  if  the  rule  succeeded,  and  a  clause 
number  if  the  rule  failed,  Indicating  which  clause  was  not  true.  EXTRA  is  T  if  the  rule 
was  found  to  fail  during  a  preview  phase  in  which  clauses  having  known  subgoals  are 
considered  before  new  subgoals  are  pursued.  For  each  rule  clause  that  is  a  compound 
predicate  function,  an  EXTRA  trace  Is  included  that  names  the  logic  or  mapping 
function  and  includes  traces  about  its  application.  Note  that  the  details  for  an  internal 


1  A.  Carlisle  Scott  helped  the  author  design  this  part  of  the  consultation  program. 
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mapping  function  are  of  the  same  format  as  the  trace  for  the  rule  as  a  whole  This  is 
because  each  mapping  function  consists  of  a  list  of  contexts  (objects)  and  an  internal 
$AND  predicate,  just  like  a  rule.  On  the  other  hand,  details  for  an  internal  logic 
function  are  provided  by  an  EXTRA  trace,  consisting  of  a  clause  number  or  information 
about  an  compound  clause. 

APPLTRACE  -  (  CNTXT  FATE  (EXTRA)*  ) 

CNTXT  -  an  object  in  the  context  tree,  e.g.,  CULTURE-1 
FATE  «-  NIL  |  C# 

C#  •-  a  clause  number,  e.g.,  C3. 

EXTRA  -  T  |  C#  |  (C#  FAIL/SUCC  MAPFN  (APPLTRACE)*  )  | 

(C#  FAIL/SUCC  LOGICFN  (EXTRA)*  ) 

FAIL/SUCC  -  FAIL  |  SUCC 

MAPFN  -  FINOMAX  |  FINDMIN  J  THEREARE  |  FORALL  |  THEREXISTS 
LOGICFN  -  SANO  |  SOR 

Figure  A.4.  Format  of  the  Consultation  Record  Trace 


Figure  A.6  shows  a  skeleton  of  a  rule  and  a  possible  APPLTRACE  that  might  be 
associated  with  It  after  a  consultation.  The  trace  indicates  that  the  rule  failed 

because  the  second  clause  was  not  true  (a1).  The  EXTRA  trace  (b)  indicates  that  the 
second  clause,  a  FORALL  iteration,  failed  because  In  the  context  of  ORGANISM-3  the 
third  clause  of  the  internal  $AND  failed  (c).  The  EXTRA  trace  associated  with  this 
third  clause  indicates  that  the  $0R  succeeded  (d).  (In  other  words,  the  SOR  was  true 
and  negated,  so  the  SAND  of  the  FORALL  was  not  true  for  ORGANISM-3,  thus  the 
second  clause  of  the  rule  fails.) 


PREMISE:  (SAND  (SAME  ...) 

(FORALL  organisms 

[SAND  (NOTSAME  ...) 

(THOUGHTNOT  ...) 

(NOT  (SOR  (SAME  ...) 

(SAME  ...)) 

(SAME  ...]) 

(SAME  ...)) 

APPLTRACE: 

( CULTURE -1  C2  (C2  FAIL  FORALL  (ORGANISM-3  C3  (C3  SUCC  SOR))) 

a  b  c  d 

Figure  A.5.  Rule  Skeleton  and  Associated  Consultation  Record 


1  Small  letters  point  to  parts  of  the  trace 


* 


I 


1 
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The  discussion  that  follows  deals  with  the  second  phase  of  the  analysis  (refer 
to  Figure  A.3).  Note  that  we  will  not  refer  to  the  APPLTRACE  again.  All  references  to 
whether  or  not  a  predicate  succeeds  or  fails  are  determined  by  decoding  the 
APPLTRACE. 


As  stated,  the  analysis  takes  place  on  three  levels.  An  overview  is  given  below, 
referring  to  program  functions  by  name.  Indentation  Indicates  that  a  function  is 
invoked  by  the  one  above  it.  Level  of  analysis  is  indicated  by  the  functional 
arguments  following  function  names. 

EXPANDNODE  —  goal 

SETUPOATE!  —  rule 

NARKSUBGOALS  --  rule 

MARK CLAUSES  -  rule 

HARKCL1  —  clause  • 

MARKMAP  —  mapping  clause 
MARKSG1  —  subgoal 

Figure  A.6.  Calling  Structure  of  Functions  for  Constructing  the  Tree 


Expanding  a  goal  node  (EXPANDNODE) 


Expansion  of  a  goal  node  (parameter/context  pair)  consists  of  analyzing  each 
rule  that  might  have  been  used  to  determine  the  goal.  One  might  view  this  as  a 
process  of  attaching  rule/context  pairs  to  the  goal  so  that  they  hang  below  it.1  For 
later  access,  these  rules  are  grouped  Into  those  that  fail  and  those  that  succeed.  So 
the  tree  we  are  building  looks  like  this: 


goal 


UPOATE-SUCC !  UPDATE-FAIL! 

Figure  A.7.  Grouping  Rules  below  a  Goal  Node 


The  format  of  the  UPDATE-SUCC!  and  UPDATE-FAIL!  properties  is  shown  in 
Section  8.3.2.2.  Each  is  a  list  of  rule/context  pairs  with  a  marker  separating  self¬ 
reference  rules  from  normal  rules.  Note  that  if  a  rule  was  not  tried  at  all  (because 
other  rules  gave  the  answer  with  certainty),  then  it  Is  put  on  UPDATE-NOTUSED!  (not 
shown  here). 


1  This  use  of  the  term  "expansion"  is  the  same  as,  and  was  motivated  by,  its 
use  for  searching  AND/OR  game  trees  [Nilsson,  1971] 


i 
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In  short,  process  of  expanding  a  goal  consists  of  going  down  the  list  of  rules  for 
the  goal  (union  of  UPDATED-BY  (consequent)  and  UPDATED-IN  (antecedent)),  and 
analyzing  each  rule  in  each  context  that  it  was  applied.  SETUPDATE!  places  each 
rule/context  pair  in  the  appropriate  group  of  rules  hanging  front  the  goal,  and  then 
invoices  MARKSUBGOAIS  to  analyze  the  rule. 


Analyzing  a  Rule  (MARKSUBGOALS) 


A  nested  history  list  is  set  up,  starting  with  OR  if  the  rule  failed  and  AND  if  the 
rule  succeeded.  As  subgoals  are  discovered,  they  ere  added  to  this  logical 
representation  of  the  rule.1  Next,  the  consultation  traces,  premise  of  the  rule,  and 
whether  or  not  the  rule  succeeded  are  passed  on  to  the  MARKCIAUSES. 


Analyzing  a  Set  of  Clauses  (MARKCLAUSES) 


If  a  SAND  succeeded,  then  all  clauses  are  analyzed  by  MARKCL1  2 3 4  so  that  all 
subgoals  are  placed  in  the  history  being  built.  If  a  SAND  failed,  only  clauses  that 
failed  have  their  subgoals  placed  in  the  history.  Subgoais  in  clauses  that  succeed  are 
unused,  and  this  information  is  passed  on  to  MARKCL1. 


The  main  problem  Is  determining  whether  or  not  a  clause  succeeds  or  fails.  If 
this  can't  be  determined  from  the  trace,  i.e.,  the  clause  was  not  evaluated  during  the 
consultation,  then  the  clause  is  evaluated  at  this  time.  Since  additional  case  data 
may  required  to  evaluate  the  clause,  It  may  not  be  possible  to  classify  it  as  failing  or 
succeeding.  In  this  case,  subgoals  In  the  clause  are  marked  "unused,"  and  the 
UNKNOWNPARMS  property  is  placed  on  the  rule  for  later  use  in  the  tutorial,  e.g,  when 

generating  a  quiz  from  the  rule.3  4 


1  A  rule  that  fails  is  represented  as  a  logical  OR  because  there  may  be  more 
than  one  clause  that  is  false. 

2  "Mark  a  single  clause" 

3  Currently,  an  Iterative  clause  can  only  be  marked  as  failing  if  it  is  the  clause 
that  is  known  to  have  caused  the  rule  to  fail,  otherwise  it  is  marked  as  "unused."  To 
do  better  than  this,  we  would  have  to  carefully  examine  the  clause  to  make  sure  that 
all  Information  Is  available  for  It  to  be  evaluated.  Even  then  other  rules  might  need  to 
be  invoked  to  supply  parameter  values.  We  have  simplified  the  Initial  implementation 
by  avoiding  this  problem.  The  effect  In  a  tutorial  Is  that  GUIDON  may  present  a  failing 
rule  after  the  student  already  knows  that  It  Is  not  applicable  (due  to  a  false  iterative 
clause  that  GUIDON  is  Ignoring). 

4  Self-referencing  rules  require  special  treatment.  When  EXPANDNODE  detects 
that  is  Is  passing  seif  referencing  rules  on  for  analysis,  it  sets  an  internal  pointer  so 
that  the  value  of  the  goal  is  the  value  that  had  been  computed  in  the  consultation 
after  all  normal  rules  had  been  done.  Thus,  the  rule  will  be  evaluated  just  as  It  was 
during  the  consultation  (see  [ShortMffe,  1870]  for  details)). 


367 


The  presence  of  a  NOT  in  the  rule  complicates  matters.  It  is  then  possible  that  a 
clause  that  succeeds  may  be  "unused"  and  one  that  fails  may  be  included  in  the 
history  of  why  the  rule  succeeded.  To  bring  about  this  inversion,  MARKCLAUSES  is 
parametrized  to  look  for  a  particular  fate  (success  or  failure),  and  this  fate  is  treated 
as  a  switch  that  is  inverted  when  NOT  is  encountered  by  MARKCL 1 . 


Marking  a  Particular  Clause  (MARKCL1) 


The  next  step  is  to  extract  the  subgoals  that  must  be  known  for  a  clause  to  be 
evaluated.  The  analysis  separates  Into  a  handful  of  cases  dependent  on  the  function 
that  represents  the  clause:  $AND,  $0R,  NOT,  mapping  function,  complex  functions,  and 
simple  functions.  In  all  cases,  an  ANO  or  OR  is  pushed  onto  the  history  list  when 
analysis  begins,  subgoals  are  added  (by  MARKSG1)  as  they  are  discovered,  and  the 
list  is  popped  (closed)  when  analysis  of  th~  clause  is  complete,  in  general,  AND  is 
pushed  when  the  clause  is  to  be  marked  "succeed"  and  the  function  is  a  conjunction 
(FORALL  and  $AND),  or  the  clause  is  to  be  marked  "fail"  and  the  function  is  a 
disjunction  (THEREXISTS  and  $0R).  OR  Is  pushed  for  failing  conjunctions  and 
successful  disjunctions.  If  the  clause  Is  within  a  NOT,  then  AND  and  OR  are  reversed. 


Analysis  of  $AND  and  $0R  involves  sending  the  clauses  of  the  conjunction  or 
disjunction  as  a  set  to  be  marked  by  MARKCLAUSES,  with  appropriate  switches  set. 
When  NOT  is  encountered,  a  flag  is  inverted  and  the  clause  it  qualifies  is  sent  to 
MARKCL  1 .  Mapping  clauses  are  analyzed  by  MARKMAP  (discussed  below).  Templates 
are  used  to  pick  out  the  parameters  mentioned  in  complex  and  simple  clauses. 
Parameters  are  paired  up  with  relevant  contexts  (determined  from  the  context  in 
which  the  rule  was  applied  by  examining  the  context  tree),  and  then  sent  to  MARKSG1 
to  be  added  to  the  history  list. 


Analyzing  a  mapping  clause  involves  first  computing  the  list  of  contexts  over 
which  the  Iteration  was  performed.  The  internal  $AND  Is  then  analyzed  in  each 
context.  If  the  clause  failed  because  there  were  no  objects  of  that  type  in  the 
consultation,  e.g.,  no  current  therapy,  then  the  parametrized  name  of  this  list  is  sent 
to  MARKSG1,  e.g.,  "current  therapies  of  patient-638"  Is  represented  internally  as  the 
pair  (CURTHER  .  PATIENT-638). 


In  general,  iterations  that  succeeded  are  so  marked  and  others  are  "unused." 
For  example,  a  THEREXISTS  might  have  evaluated  the  internal  predicate  ($AND)  in 
several  contexts  (ORGANISM- 1,  ORGANISM-2...)  before  a  true  case  was  found.  The 
true  $AND  Is  sent  to  MARKCL  1  to  be  marked  so  that  each  subgoal  is  added  to  the 
successful  history  of  the  rule.1  Again,  this  analysis  is  inverted  if  a  NOT  surrounds  the 


1  Note  that  a  successful  THEREXISTS  clause  should  be  recorded  in  the  form:  (OR 
(AND  ...)  (AND  ...) ...)  In  which  there  la  one  AND  for  each  successful  iteration.  However, 
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mapping  clause.  Note  that  all  contexts  (Iterations)  of  a  pure  iterative  function  are 
marked  identically. 


Entering  a  Subgoel  into  the  AND/OR  Tree  (MARKSG1) 


At  this  point  we  have  parameter/context  pair  and  a  mark:  SUCCEED,  FAIL,  or 
UNUSED.  Moreover,  this  mark  Indicates  (if  we  have  done  all  of  our  inversions 
correctly)  whether  the  rule  succeeds  or  fails,  or  is  not  used  at  all.  If  the  mark  is 
SUCCEED,  the  rule  is  entered  on  the  USE-SUCC!  property  of  the  subgoal  (rules  that 
succeed  that  use  the  subgoal)  (refer  to  Figure  4.5).  The  subgoal  is  placed  on  the 
CONSEQ-SG!  or  ANTE-SG!  property  of  the  goal  that  the  rule  updates  (this  is  a  global 
variable  set  by  EXPANDNODE).  If  the  mark  is  FAIL,  the  ^ule  is  added  to  the  CAUSE - 
FAIL!  property  of  the  subgoal  and  the  subgoal  is  added  to  the  FAIL-SG!  property  of  the 
goal.  If  the  mark  is  UNUSED,  the  relevant  properties  are  UNUSED-PARMS!  and 
UNUSED-SG!.  Inverse  pointers  from  subgoal  to  goal  have  not  been  found  to  be  useful. 


Finally,  if  the  mark  is  not  UNUSED,  the  subgoai  is  added  to  the  logical 
representation  of  the  rule.  When  MARKCLAUSES  has  analyzed  the  entire  rule, 
MARKSU0GOALS  will  save  this  history  as  the  SUCC-PARMS!  or  FAIL-PARMS!  property  of 
the  rule. 


we  only  mark  the  single  (first)  iteration  that  succeeded  (and  the  rest  are  UNUSED). 
This  is  because  we  do  not  evaluate  iterations  that  MYCIN  did  not  attempt  (because 
other  rules  would  probably  need  to  be  invoked  to  do  this).  The  effect  during  a  tutorial 
is  that  a  student  might  know  that  the  THEREXISTS  succeeds,  based  on  a  later 
iteration  that  MYCIN  never  tried,  while  GUIDON  will  believe  that  it  is  too  early  to  fire 
the  rule  because  there  Is  Insufficient  evidence  to  evaluate  the  only  successful 
iteration  that  it  knows  about.  The  best  reason  for  Ignoring  this  problem  for  now  is  that 
there  are  In  general  a  small  number  of  possible  Iterations,  which  reduces  the 
possibility  of  there  being  multiple  iterations  that  are  true. 
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Appendix  B 

Question  Generation  and  Response 


A  basic  feature  of  GUIDON  is  its  ability  to  generate  questions  from  MYCIN  rules. 
There  are  four  distinct  steps  in  this  process: 

1 )  Selecting  a  question  type 

2)  Converting  the  rule  and  printing  the  question 

3)  Evaluating  the  student's  answer 

4)  Responding  to  the  student. 


A  question  format  is  described  in  terms  of  the  format  of  the  premise  part  of  the 
question  and  the  format  of  the  action  or  conclusion  part.  There  are  10  possible 
premise  formats  and  6  possible  action  formats,  yielding  19  valid  combinations  that 
have  been  implemented.  Because  of  the  number  of  question  formats  and  complications 
deriving  from  features  of  the  rules,  the  code  for  generating  questions  is  very  complex. 
This  appendix  describes  the  valid  question  formats,  relevant  rule  features,  and 
strategies  for  responding  to  the  student. 


Question  Formats 


Question  formats  fall  into  two  groups,  13  valid  combinations  that  are  selected 
randomly  and  6  special  purpose  formats  used  to  respond  to  a  student's  hypothesis  or 
present  a  rule.  The  randomly  chosen  formats  are  summarized  in  Figure  B.l.  A  dollar 
sign  ($)  separates  the  premise  and  action  parts  of  the  question. 
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\PREHISE 

A\ 

c  \ 

T  \ 

I  \ 

0  \ 

N  \ 

FACTS 

RIGHT-HYPOTHESIS 

or 

WRONG-HYPOTHESIS 

MULTIPLE  CHOICE 

ACTUAL 

VALUE 

"Given  Cprem  facts>, 
does  this  S  tell  you 
<actual  concl>?" 

"Given  <prem  facts), 
if  Chypothesis), 
would  this  i  tell 
you  <actual  concl)?* 

"Given  Cprem  facts), 
what  Chypothesis) 
would  S  tell  you 
Cactual  concl)? 

Select  from  1 )  ... 

N)" 

WRONG 

VALUE 

same  as  ACTUAL  VALUE 

same  as  ACTUAL  VALUE 

(invalid  for 
WRONG-HYPOTHESIS) 

Invalid 

MULTIPLE 

CHOICE 

"From  <prem  facts) 
one  can  $  conclude 
that  <goal>  is: 

Select  from  1 )  ... 

N)" 

"Given  <prem  facts), 
if  Chypothesis),  one 
could  S  conclude 
that  <goal)  is: 

Select  from  1 )  ... 

N)" 

invalid 

CONFIRM 

"Do/Does  <prem  facts) 
$  tend  to  confirm  the 
conclusion  that 
<other  rulers 
concl >7" 

"Given  <prem  facts), 
if  Chypothesis), 
would  this  S  tend 
to  confirm  the  con¬ 
clusion  that  Cother 
rulers  cone  1)7" 

"Given  Cprem  facts), 
what  Chypothesis) 
would  S  tend  to 
confirm  the  con¬ 
clusion  that  Cother 
rulers  concl)?  Select 
from  1)  ...  N)" 

Figure  B.l.  Randomly-selected  Question  Formats 

Key:  <prem  facts>  *  listing  of  case  data  used  by  clauses 
<actual  concl)  =  conclusion  made  by  the  rule 
<hypothesis>  *  certain  clauses  of  the  rule  stated  hypothetically 
<goal>  «  parameter  to  be  determined 
<other  rule's  concl)  *  conclusion  made  by  another  rule 
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The  six  special  formats  are  given  below  with  comments.  ("S"  replaces 
"student.") 


\ 


PREMISE/ACTION 

FORMAT 

COMMENTS 

INVAL IDF ACTORS/ 
ACTUAL  VALUE 

"What  facts  about  this  case 
tell  you  that  (actual 
concl >?" 

Requests  S  to  support 
his  hypothesis  by  stating 
relevant  factors.  None 
will  be  accepted. 

KNOWNFACTORS/ 

ACTUAL  VALUE 

sane  as  INVALIOFACTORS 

Quiz  based  on  multiple 
rules.  Many  factors 
might  be  valid.  Schema 
(context)  factors  are 
accepted,  but  not 
required. 

NEWF ACTORS/ 

ACTUAL  VALUE 

■What  factors  would  S 
tell  you  that  (actual 
concl)?" 

S  is  asked  to  mention 
factors  that  haven^t  been 
supplied  for  this  case 
yet.  Used  to  present  a 
rule. 

NEWSUBGOALS/ 

ACTUAL  VALUE 

sane  as  NEWFACTORS 

Here  the  factors  must  be 
established  by  applying 
other  rules.  Used  to 
introduce  discussion  of 
the  rule  that  uses  these 
subgoals. 

-  -  - 

GIVENFACTS/ 

FILLIN 

■Here  Is  some  relevant 
data  you  could  have  asked 
for:  (case  data).  S  What 
does  this  tell  you  about 
(goal >7" 

For  quick  presentation 
of  new  data  and  a  rule 
that  uses  it. 

KEYFACTS/ 

FILLIN 

■Consider  (key  facts). 

S  What  does  this  tell  you 
about  (goal)?" 

This  Is  a  standard  format 
for  testing  the  student''* 
knowledge  of  rules  that 
have  succeeded. 

Figure  B.2.  Special-purpose  Question  Formats 
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Selecting  Question  Formats 


The  rule  packet  GENOTYPE. RULEKP024  determines  which  formats  are  loqicaiiy 
valid  for  a  given  d-rule,  and  chooses  randomly  from  the  candidates.  Besides 
interactions  between  the  premise  and  action  formats,  the  most  relevant  rule  features 
are  the  outcome  of  the  rule's  application  (success  or  failure)  and  the  presence  of 
true/false  parameters.  No  attempt  will  be  made  here  to  describe  the  kinds  of 
interactions  we  have  found;  the  interested  reader  will  find  them  in 
GENOTYPE. RULEPK024. 


It  is  important  to  note  that  randomly  generating  question';  i>»  '-'ving  rules  is 
risky.  Resulting  questions  can  be  trivial,  or,  even  worse,  a  nn«ti(i>  .  ■>,  jut  produce 
another  correct  rule  (e.g.,  see  how  T-RULE24.02  handles  this).  Even  without 
modifying  a  rule.  GUIDON  can  run  into  trouble  if  the  student  is  asked  to  fill  in  an 
answer  or  choose  from  all  possible  values.  Consider  the  following  quiz-. 

PREMISE;  multiple  choice 
ACTION:  actual  value 

Given  the  infection  which  requires  therapy,  what  value  for  the  diagnoses 
which  are  consistent  with  Pt538‘s  clinical  history  would  tell  you  that  Virus 
is  one  of  the  organisms  (other  than  those  seen  on  cultures  or  smears)  which 
might  be  causing  the  infection  (.2)? 

(select  from  the  following): 

1)  OTITIS-MEDIA 

2)  EPIGLOTTITIS 

3)  MUMPS 

4)  LEPTOSPIROSIS 

5)  COCCIDIOIDOMYCOSIS 

**  3 

No,  you  could  make  this  conclusion  if  epiglottitis  were  one  of  the 
diagnoses  which  are  consistent  with  Pt538’s  clinical  history  [ RULE588 ] . 

Figure  B.3.  Question  with  Unrecognized  Correct  Answer 


It  so  happens  that  MUMPS  is  a  correct  answer  that  can  be  deduced  from 
MYCIN's  rule  set,  but  Is  not  stated  directly  In  this  rule.  (In  particular,  there  is  a  rule 
"MUMPS  =>  type  of  infection  is  VIRAL"  and  another  rule  that  states  "VIRAL  type  => 
VIRUS  might  be  causing  the  infection.")  Part  of  the  problem  here  is  that  parameters 
are  not  always  used  consistently  and  related  inferences  are  separated  into  several 
rules.  When  disallowing  an  inference  of  the  form  "A  =>  B,"  GUIDON  really  needs  to  look 
beyond  the  single  rule  it  is  quizzing  about,  to  prove  that  there  isn’t  another  rule  of  this 

form  1  or  to  prove  that  there  isn't  a  chain  of  reasoning  that  ties  A  to  B,  as  shown  in 
the  example  here. 


1  Often  analogous  to  the  rule  being  quizzed  about,  say  C  »>  B,  where  C  and  A 
are  in  the  same  class,  e.g.,  kinds  of  rashes. 
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Modifying  the  rule 


Modifying  the  rule  so  that  it  can  be  printed  as  a  question  requires  function 
templates,  the  list  of  parameter/context  pairs  that  caused  the  rule  to  succeed  or  fail, 
the  key  factors  of  the  rule,  alternative  values  for  parameters  used  in  the  rule,  and  the 
actual  conclusions  made  by  the  rule,  if  it  succeeded.  To  make  the  question  more 
concise,  premise  factors  are  printed  in  general  terms  ("information  about  the  type  of 
the  infection")  rather  than  in  full  detail  ("the  fact  that  the  type  of  INFECTION-1  is 
bacterial  (.75)"),  and  only  key  factors  are  mentioned.  Ad  hoc  code  changes  verbs  to 
plural  form,  converts  verbs  to  subjunctive  form,  inserts  punctuation,  and  capitalizes 
initial  words.1  Random  values  and  certainty  factors  are  used  to  create  "wrong” 
premise  and  action  parts  to  the  question.  Values  are  chosen  from  among  those  that 
actually  appear  in  MYCIN  rules  so  that  they  are  plausible,  and  certainty  factors  are 
shifted  enough  to  be  meaningfully  different,  without  being  obviously  wrong. 


It  would,  of  course,  be  useful  to  generate  questions  that  make  a  point,  as  is 
done  in  WHY.  Tabular  rules  offer  more  opportunity  for  generating  meaningful  questions 
because  premise  and  action  are  related  by  a  case  statement  that  pairs  alternative 
situations  with  appropriate  Inferences.  This  is  a  convenient  representation  for 
generating  hypothetical  questions.  Though,  again,  without  the  logical  underpinnings 
that  justify  the  rule's  conclusions,  GUIDON  cannot  quiz  the  student  about  the 
principles  Involved  (the  point  of  Section  6.2). 


Two  other  points  about  GUIDON'S  question  generation  are  worth  mentioning. 
"DONTKNOW"  is  always  a  valid  reply;  it  is  treated  like  "HELP,"  which  causes  a  fill-in 
question  to  be  converted  to  multiple-choice  format.  WRONG-HYPOTHESIS  and  RIGHT- 
HYPOTHESIS  involve  restating  a  clause  in  subjunctive  form  that  caused  the  rule  to  fail. 
RIGHT-HYPOTHESIS  prints  the  clause  as  it  appears  in  the  rule;  WRONG-HYPOTHESIS 
replaces  the  value  of  the  clause  with  another  possible  value,  e.g.,  "petechial  rash”  is 
changed  to  "purpuric  rash." 


Evaluating  the  Student's  Answer 


During  question  generation,  flags  and  variables  are  set  to  indicate  the  nature  of 
the  question  ("multiple  rules  flag,"  "original  tabular  case,"  "correct  values").  These 
are  used  in  a  straightforward  way  to  logically  evaluate  the  student's  reply. 


1  William  van  Melle's  improvements  to  MYCIN's  string  manipulation  and  printing 
functions  were  very  helpful. 


4 


t 
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Responding  to  the  Student 


Code  for  replying  to  the  student  is  as  complex  as  that  used  for  generating  a 
question,  for  it  again  breaks  down  into  19  separate  cases  involving  different  data 
structures.  The  basic  response  strategy  is  to  list  right,  wrong  and  missing  answers, 
and  state  what  could  have  been  concluded  given  the  facts  stated,  or  when  the 
conclusion  stated  would  be  true.  If  the  rule  has  been  modified,  details  are  provided 
even  when  the  student  is  correct.  The  rule's  key  factor  is  mentioned  if  it  has  not 
been  distinguished  in  the  question. 


Ideally,  these  tutorial  actions  should  be  stated  in  tutorial  rules  to  make  them 
more  accessible  for  change.  However,  most  of  the  strategies  are  implicit  in  print 
statements  that  go  through  many  contortions  to  combine  fixed-text  with  translations 
of  parameters  and  values.  This  problem  has  not  been  solved  in  t-rules  either,  where 
"SAY-."  with  a  tag  indicates  a  print  statement.  A  higher  level  specification  language 
would  be  useful  here. 


Summary  of  the  Questioning  Strategy 


GUIDON'S  1 9  formats  for  generating  questions  can  be  classified  according  to  the 
Socratic  probing  and  correcting  tutorial  methods  formalized  by  Collins  and  Stevens 
[Stevens  &  Collins,  1977].  The  formats  of  Figure  B.1  correspond  to  "ask  the  student 
to  form  or  verify  the  correctness  of  a  rule."  Figure  B.2  contains  two  methods  (split  by 
the  double  line):  "Ask  for  relevant  factors"  and  "Ask  for  the  value  of  a  dependent 
variable".1 


Generally,  GUIDON'S  distinctions  about  what  method  to  apply  (its  questioning 
strategy)  is  based  on  the  convenience  of  fitting  a  given  rule  to  a  particular  format  and 
the  assumption  that  Figure  B.2  questions  are  more  difficult  than  Figure  B.1  questions. 
Thus,  the  strategy  is  based  on  variety,  difficulty,  and  the  dialogue  situation  (what  has 
been  discussed).  The  main  exception  occurs  during  response  to  a  student's 
hypothesis.  Then,  selection  of  question  type  parallels  the  Collins/Stevens  model,  e.g., 
when  debugging  an  Incorrect  student  hypothesis,  GUIDON  asks  the  student  for  prior 
factors  that  support  his  value  for  the  dependent  variable  (KNOWN  or  INVALID  factors 
format).  At  other  times  in  the  dialogue,  the  Collins/Stevens  enumeration  of 
questioning  strategies  would  be  useful  if  we  wanted  GUIDON  to  follow  up  on  wrong 
student  replies  to  questions,  rather  than  always  correcting  the  student  by  a 
statement.  For  example,  when  the  student  mentions  insufficient  factors,  the  program 
could  form  a  hypothetical  question  to  point  out  the  missing  factors,  instead  of  stating 
them. 


1  Note  that  the  distinction  among  invalid,  known,  and  new  factors  and  between 
given  and  key  factors  Is  based  on  whether  or  not  the  rules  involved  can  be  applied  at 
this  point  in  the  dialogue;  these  formats  do  not  represent  different  methods. 
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Appendix  C 
GUIDON  Handbook 


This  is  a  condensed  form  of  an  operating  manual  for  GUIDON.  This  handbook  is 
generated  by  a  program  from  text  files  that  serve  as  program-aids  during  the  tutorial. 
Note  that  the  text  of  section  VII  is  specific  to  the  rule  base  on  which  GUIDON  is  built. 
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I.  What  is  GUIDON? 

GUIDON  Is  a  tutorial  program  that  can  discuss  particular  problems 
(cases)  with  a  student  in  mixed-initiative  dialogue.  A  "diagnostic  game"  is 
played;  the  student  is  given  some  Information  about  the  problem  (the  "case") 
and  must  pursue  other  Information,  leading  to  solution  of  subgoals 
and  a  final  hypothesis  about  the  problem  (a  "diagnosis").  While  the  tutor 
adheres  to  a  goal-directed  discussion,  the  student  can  change  the  topic 
under  discussion  at  any  time. 

GUIDON  has  access  to  a  fully  worked-out  solution  of  the  case.  It 
uses  the  solution  as  a  basis  for  offering  assistance  at  any  time,  making 
hints  about  conclusions  the  student  should  be  reaching,  and  interpreting  and 
responding  to  student  hypotheses  about  relevant  subgoals.  In  addition,  a 
wide  variety  of  options  are  available  to  the  student  that  make  it  possible 
for  him  to  get  an  overview  of  the  given  case  and  explore  subgoals  according 
to  his  interests.  For  example,  the  student  " profile "  can  cause  GUIDON  to 
indicate  when  a  final  conclusion  about  a  subgoal  can  be  drawn,  and  to  mention 
when  any  conclusion  at  all  about  the  current  topic  can  be  drawn. 

It  is  assumed  that  the  student  is  familiar  with  types  of  domain 
specific  "objects"  and  "measurements"  involved  in  solving  a  case.  For 
example,  if  a  case  dealt  with  diagnosis  of  car  horn  failures,  it  is  assumed 
that  the  student  would  know  what  a  car  horn  is  and  realize  that  mechanical 
and/or  electrical  principles  are  involved  in  failures.  The  instructional 
objective  of  GUIDON  is  to  teach  the  student  METHODS  for  reaching  conclusions 
about  the  problem  (e.g.,  a  rule  for  relating  horn  symptoms  to  possible 
faults)  and  STRATEGIES  for  applying  these  methods. 

The  sections  below  duplicate  most  of  the  text  that  is  available 
online  to  a  student  (rules  appear  in  a  separate  document).  It  is  presented 
here  to  serve  as  an  introduction  to  the  program  and  for  routine  reference 
during  a  session. 


II.  The  Role  of  the  Student  and  His  Options. 

Brief  Description  of  the  Tutorial  Session 

You  are  to  play  the  role  of  consultant  for  a  patient  suspected  to 
have  an  infection.  After  Initial  data  has  been  presented,  you  should  request 
more  information  about  the  patient,  decide  which  organisms  (if  any)  require 
therapy,  and  prescribe  treatment.  The  tutor  will  direct  you,  according  to 
the  profile  you  set.  Its  behavior  can  vary  from  leaving  you  alone  until  you 
are  ready  to  make  a  diagnosis,  to  directing  your  thinking  at  every  step. 

Feel  free  to  change  your  profile  or  request  HELP  to  solve  the  problem  at  any 
time.  Use  a  question  mark  (?)  whenever  you  want  to  see  the  list  of  options 
open  to  you.  Remember  that  the  purpose  of  this  session  is  to  present  MYCIN's 
rules  of  Infectious  disease  diagnosis  and,  in  the  second  part,  the  criteria 


« 
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of  therapy  selection. 

Type  controt-G  to  call  GUIDON.  You  will  receive  the  initiative  after 
the  next  question  is  asked. 

III.  The  Student  Profile 

The  user  profile  is  a  set  of  flags  that  are  used  by  the  EXPERT 
program.  As  the  student  receives  more  data  about  a  case,  tht  EXPERT  applies 
domain  rules  that  use  the  information  and  draws  new  conclusions.  These 
flags  can  be  set  to  control  what  the  EXPERT  tells  the  user  about  these 
rules.  A  special  flag  for  mentioning  rule  conclusions  can  be  set  for 
"relevant"  conclusions  only,  so  rules  that  aren't  relevant  to  the  topic 
under  discussion  won't  be  mentioned.  The  PROFILE  option  permits  the 
student  to  change  the  settings  of  these  flags  at  anytime,  so  experimentation 
is  encouraged. 


The  standard  profile  setting  is  for  a  goal-directed  dialogue.  The 
expert  indicates  when  a  final  conclusion  can  be  drawn  for  each  topic,  and 
mentions  when  any  conclusion  at  all  can  be  drawn  about  the  current  topic. 
The  student  is  assumed  to  be  a  novice  (1  on  scale  from  0  to  4). 


{--  omitted  here;  see  Section  7.1.3  — } 
Meaning  of  each  profile  flag: 


{—  omitted  here;  see  Figure  7.1  — } 


IV.  Option  Summary 
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V.  One  Line  Restatement  of  Options 
■  Options  availabie  during  discussion  of  a  topic: 
OPTION  INTENT 


Options  for  Determining  Dialogue  Context 

RULE  "What  rule  are  we  discussing?" 

TOPIC  "What  topic  we  are  discussing?" 

ORIENT  "Where  are  we?  How  did  we  get  here?  Where  are  we 

going?" 

Options  for  Acquiring  Case  Data 

DATA  "Give  me  this  case  data." 

BLOCK  <b1ock  name>  "Give  me  all  of  the  case  data  In  this  block." 

ALLDATA  "Tell  me  all  of  the  case  data  relevant 

to  this  topic." 

Options  for  Getting  Direct  Assistance 

HELP  "Suggest  something  for  me  to  consider." 

HINT  "Give  me  a  hint." 

TELLME  "I  give  up;  tell  me  the  conclusion  the  expert  made." 


Options  for  Information  Retrieval 

USE  "How  can  apply  Information  about  this  topic  to  this 

problem?" 

CONCLUDE  <value>  "How  can  I  conclude  that  the  current  topic  Is  this 
value?" 

SUMMARIZE  "List  the  data  and  conclusions  that  the  expert  made." 

DETAILS  <remark  1ndex> 

"Give  me  details  about  this  remark." 

FACTORS  "What  case  data  and  subgoals  should  I  consider?" 

SUBGOALS  "Show  me  a  tree  of  goals  that  we  will  pursue." 

PENDING  "What  case  data  and  subgoals  are  still  pending  for 

this  topic,  and  for  which  goals  should  I  try  to 
make  an  hypothesis?" 

Options  for  Conveying  What  You  Know 

IK NOW  "I  know  this." 

HYPOTHESIS  "I  want  to  tell  GUIDON  a  conclusion  I  have  drawn." 

Options  for  Changing  the  Topic 

DISCUSS  "Let’s  discuss  this  topic." 

QUIZ  "Ask  me  a  question." 

ALLRULES  "I  don’t  want  to  ask  any  more  questions  about  this 

topic;  let's  discuss  the  relevant  rules." 

QA  "I've  got  a  general  question  for  you." 
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control-G  "Stop  asking  me  questions;  I  want  to  say  something." 

Special  Options 

?  "What  can  I  do  here?" 

OPTIONS  "Show  me  a  table  that  summarizes  my  options." 

<opt1on>  ?  "Tell  me  more  about  this  option." 

COMMENT  "I  want  to  send  a  message  to  the  person  In  charge!" 

DEFINE  "Give  me  a  definition  of  this  word." 

PR  <rule  no.>  "Show  me  this  rule." 

JUSTIFY  <rule  no.>  "Justify  this  rule." 

LITERATURE  <rule  no.>"G1ve  references  for  this  rule." 

PTR  <T-rule  no.>  "Show  me  this  tutorial  rule." 

PROFILE  "I  want  to  change  what  the  Expert  tells  me." 

STOP  "I  want  to  stop  now.  Terminate  the  dialogue." 


When  none  of  these  options  Is  specified,  the  tutor  assumes  that 
you  are  asking  for  a  piece  of  data,  l.e.,  the  DATA  option. 

Remember:  HELP  Is  a  request  for  assistance  to  solve  the  problem; 

?  prints  your  options;  use  this  when  you  aren’t  sure  what 
Is  a  meaningful  reply  or  action  to  take.  Also,  follow 
an  option  name  by  *?"  If  you  want  details  about  It. 


{—  omitted  here  — } 


VI.  detailed  Description  of  Each  Option 
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VII.  Parameters  of  the  MVCIN  System 

Below  are  listed  all  of  the  possible  topics  that  can  be 
discussed  during  a  MYCIN/GUIDON  tutorial.  They  are  grouped  by  the 
"object"  that  Is  qualified.  In  a  given  case,  there  may  be  more  than 
one  object  of  each  type,  e.g.,  more  than  one  culture,  so  we  will 
refer  to  CULTURE- 1,  CULTURE-2,  etc. 

For  many  of  the  options  you  will  find  it  easier  to  use  the 
abbreviated  name  of  the  parameter,  e.g.,  ABDOWOUND  in  the  command: 
"DISCUSS  ABDOWOUND."  Alternatively,  you  can  also  use  phrases  like 
"How  is  the  fact  that  drugs  were  given  adequately  to  the  patient  relevant 
in  this  case?" 


{At  this  point,  the  parameters  are  listed,  grouped  by  culture,  organism,  patient,  etc. 
For  MYCIN,  this  listing  requires  six  pages.  Here  we  only  show  the  parameters 
that  appear  in  Figure  5 .20.) 


A.  The  Parameters  Associated  with  a  patient  are: 


AGE: 

ALCOHOLIC : 
ANIMALEXP: 
BURNED: 

CARDIACSURGERY : 


The  age  of  the  patient 
Whether  the  patient  is  an  alcoholic 
patient’s  age  group 

Whether  the  patient  has  frequent  contact  with 
animals 

Whether  the  patient  has  been  seriously  burned 


Whether  the  patient  has  had  cardiac  surgery 
Whether  the  results  of  a  recent  CBC  of  the 
patient  ere  available 

CL INEV- TUBERCULOSIS: 

Whether  Tb  Is  one  of  the  diagnoses  consistent  with 
the  patient's  clinical  history 
Whether  there  Is  evidence  of  congenital 

malformation  involving  the  central  nervous 
system 

Whether  the  patient  has  undergone  recent 
radiation  therapy  to  the  CNS 
Whether  the  patient  does  live  In  a  crowded 
environment 

Whether  the  patient's  X-ray  Is  abnormal 
Whether  the  patient  has  received  cytotoxic  drugs 
just  prior  to  or  during  this  infection 
Whether  the  patient  is  a  diabetic 


CBC: 


CNSHALFORN : 


CNSRADIATE: 

CROWD: 

CXRAB : 
CYTOTOXIC: 


DIABETIC: 
DIAGNOSES-LYMPHOMA: 


Whether  lymphoma  is  one  of  the  diagnoses  of  the  patient 
DIA6N05ES- LEUKEMIA: 

Whether  leukemia  Is  one  of  the  diagnoses  of  the  patient 
EPIDEMIC:  The  epidemic  diseases  to  which  the  patient  has 

bean  exposed  recently 
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EPIGLOTTITIS: 

EXPOSURE: 

GU-MANIP: 

HEADINJDEF: 

HEADTRAUHA: 

JUNKIE: 

MUMPS YM: 

NEUROSURGERY: 

NOSOCOMIAL: 

OCNERVE : 

OTITIS-MEOIA: 


RACE: 

RASHES: 

SICKLECELL: 

SKININFECT: 

SPLENECTOMY: 

STEROIDS: 

TBRISK: 

WBC: 


Whether  epiglottitis  Is  one  of  the  diagnoses  consistent 
with  the  patient's  clinical  history 
The  contagious  diseases  to  which  the  patient  has  been 
exposed  recently 

Whether  the  patient  has  had  a  genito-ur inary 
manipulative  procedure 

Whether  the  patient  has  a  head  injury  defect 
Whether  the  patient  has  had  a  severe  head  trauma 
Whether  the  patient  has  a  history  of  illicit 
intravenous  drug  use 

Whether  the  patient  has  shown  symptoms  of  mumps 
Whether  the  patient  has  had  neurosurgery 
Whether  the  infection  was  acquired  while  the 
patient  was  hospitalized 

Whether  the  patient  has  evidence  of  ocular  nerve 
dysfunction 

Whether  otitis-media  is  one  of  the  diagnoses  consistent 
with  the  patient's  clinical  history 
The  race  of  the  patient 
Whether  the  patient  has  a  rash  or  cutaneous 
lesions 

Whether  the  patient  has  sickle  cell  disease 

Whether  the  patient  has  evidence  of  serious  skin 

or  soft  tissue  infection 

Whether  the  patient  has  had  a  splenectomy 

Whether  the  patient  has  received  corticosteroids 
just  prior  to  or  during  this  Infection 
Whether  the  patient  has  a  tb  risk  factor 
The  white  count  from  the  patient’s  peripheral  CBC 
(in  thousands) 
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Appendix  D 

Introductory  Orientation  for  New  Students 


As  a  form  of  introduction  for  a  new  student,  GUIDON  prints  the  goal  rule  of  the 
expert  system  and  a  sketch  of  subgoals  at  the  start  of  the  dialogue  (following  t- 
rulel.04).  The  text  shown  below  was  extracted  from  the  sample  protocol  (Section  3). 
See  Section  6.2.3.1  for  discussion. 


Praaantation  of  the  Goal  Rule 


In  order  to  determine  the  therapeutic  regimen  of  J. Smith  we  will  want  to 
consider: 

1)  A:  organisms  Isolated  from  J. Smith,  and 

B:  organisms  noted  on  smears  of  J. Smith,  and 
C:  negative  cultures  of  J. Smith,  and 

D:  suspected  Infections  without  microbiological  evidence,  and 
E:  current  drugs  of  J. Smith,  and 
F:  prior  drugs  of  J. Smith,  and 

2)  the  organisms  which  require  therapy,  and 

3)  A:  the  organisms  (other  than  those  seen  on  cultures  or  smears)  which 
might  be  causing  the  Infection,  and 

B:  whether  the  organisms  isolated  from  the  pending  csf  culture 
(CULTURE-1)  should  be  considered  for  therapy,  and 
C:  the  associated  Infections  for  which  therapy  should  cover,  and 
D:  the  organisms  that  will  be  considered  to  be  causing  the  meningitis 
(INFECTION-1)  for  the  purposes  of  therapy  selection 
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Presentation  of  Major  Subgoat* 


Sketch  of  the  tree  of  subgoals  for  detemlning  the  therapeutic  regimen  of 
J. Smith: 

7a.  The  therapeutic  regimen  of  J. Smith 

7b.  The  organisms  (other  than  those  seen  on  cultures  or  smears)  which 
might  be  causing  the  Infection 
7c.  Whether  J. Smith  has  a  head  injury  defect 


7d.  The  Infection  which  requires  therapy 


7e.  The  type  of  the  Infection 


7f .  The  diagnoses  of  J. Smith 


7g.  Whether  organisms  were  seen  on  the  stain  of  the  pending  csf 
culture  ( CULTURE- 1) 


7h.  Whether  the  organisms  Isolated  from  the  pending  csf  culture 
(CULTURE-1)  should  be  considered  for  therapy 


71.  The  organisms  that  will  be  considered  to  be  causing  the  meningitis 
(INFECTION-1)  for  the  purposes  of  therapy  selection 


(You  can  produce  a  full  tree  of  subgoals  for  any  topic  by  the  SUBGOALS 
option. ) 


Appendix  E 


Representation  of  Discourse  Procedures 


One  purpose  of  GUIDON  is  to  provide  a  framework  for  testing  teaching  methods. 
Therefore,  it  is  advantageous  to  have  a  representation  that  makes  it  convenient  to 
display  and  vary  the  strategies  that  the  tutor  uses  for  guiding  the  dialogue.  Using 
methods  similar  to  those  used  In  knowledge-based  programs,  we  have  formalized  the 
tutorial  program  in  rules  and  procedures  that  codify  tutorial  expertise.  Here  we  will 
consider  how  the  INTERLISP  representation  of  t-rules  satisfies  the  design  criteria  of 
transparency  and  flexibility. 


Example  of  Internal  Representation 


Figure  E.1  illustrates  the  Internal  form  of  a  t-rule  for  presenting  a  (successful) 
d-rule  to  the  student.  The  domain  rule  has  already  been  selected  and  variables  have 
been  set  that  describe  it  (e.g.,  FACTORSTODO,  the  number  of  case  data  factors 
mentioned  In  the  rule's  premise  that  remain  to  be  asked  by  the  student).  SUBGOALPC 
is  a  pair  of  the  form  «parameter>  .  <context>)  that  was  set  when  the  rule  was 
analyzed;  CURGOALPC  is  the  topic  currently  being  discussed.  Notice  that  substep  ii. 
sets  a  flag  when  the  GOAL  procedure  is  invoked;  this  flag  tells  GOAL  not  to 
reintroduce  the  new  topic  (see  Figure  4.9).  The  description  of  syniux  below  will  be 
general  at  first;  details  will  gradually  accomodate  this  specific  example. 


[RULE  31,04  (IF  (AND  (ZEROP  FACTORSTODO) 

(EQP  S6ST0D0  1)) 

THEN  (PROGN  (SUBSTEP\\  1.  (SAT\\  (CLISP  <"Try  to  determine" 

!  (CPTRANS!  SUBGOALPC) 
*X.  EOL  >1 
"SUGGESTSG) ) 

(SUBSTEPW  it.  (DISCUSSW  GOAL.PROC801  SUBGOALPC 

CURGOALPC  T)) 

(SUBSTEPW  Hi.  (DISCUSSW  ENDOFRULE . PR0C81 7 

RULEFOCUS  CURGOALPC))) 
(•  Subgoals  are  obvious,  he  must  not 
know  about  this  one)] 

Figure  E.1.  INTERLISP  Form  *>f  a  Sample  T-rule 


Syntax  of  Discourse  Procedures 


As  shown  in  Figure  6.1,  the  action  of  a  t-rule  is  a  discourse  procedure.  Figure 
E .2  shows  the  general  syntax  of  a  discourse  procedure. 
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(PROG  (  (local  variables)  ) 
(STEP\\  1  ...) 

(STEP\\  2  ...) 


(STEP\\  (n>  ...)) 

Figure  C.2.  Syntax  of  a  Discourse  Procedure 


The  steps  (shown  as  above)  are  compiler  macros  listed  In  Figure  E.3. 
Symbolic  Form  Operation 


(DISCUSSW  (discourse  procedure)) 

(STRATEGYW  <rule  packet)) 

(G0\\  (label)) 

(RETURNW  (value)) 

(COND\\  <1f-part> 

(then-pert)  (else-pert)) 

(FOR\\  (set) 

(primitive  function) 

(free  variable  name)) 

(SAY\\  (remark)  (tag)} 

(A5*C\\  (question)  (tag)  (yes/no-f 

Figure  E.3.  Primitive  Functions  i 


invokes  another  procedure 
invokes  a  packet  of  t-rules 
■go  to" 

return  from  procedure 

simple  conditional  nested 
within  a  t-rule 

iterative:  for  all  members 
of  the  set,  do  the  function 

prints  remark  for  student 
}>)  asks  student  a  question 
Discourse  Procedures 


In  addition,  one  of  two  special  operators  can  appear  in  place  of  a  procedure 
step:  a  debugging  statement  (DEBUG\\  (message))  prints  information  on  the  terminal 
and/or  a  trace  file  and  an  initialization  statement  (INIT\\  (arbitrary  INTERUSP  code)) 
is  used  to  set  flags  and  counters  used  by  the  t-rules.  Notice  that  every  print 
statement  (SAY\\)  is  uniquely  tagged;  this  is  useful  for  debugging. 


% 
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Syntax  of  T-rule  Packets 


T-rule  packets  are  also  INTERLISP  procedures,  however  they  generally  consist 
of  a  single  step  that  has  an  ordered  list  of  t-rules  of  the  form  (DO-ONE\\  <list  of  t- 
rules>)  or  (DO-ANY\\  <list  of  t-rules».  The  first  stops  trying  t-rules  as  soon  as  one 
succeeds;  the  second  will  apply  any  t-rule  whose  premise  is  satisfied.  T-rule  packets 
always  accomplish  a  single  task,  e.g.,  presenting  a  predetermined  d-rule.  A  t-rule  is  an 
INTERLISP  record  of  the  form: 

(RULE  <number> 

(IF  <1f-part>  THEM  <then-part>  ELSE  <e1se-part>) 

<comment>  ) 


In  practice,  it  is  often  convenient  for  a  t-rule  packet  to  return  a  value,  so  they 
are  written  with  two  steps,  one  with  DO-ONE\\  or  DO-ANY\\  and  a  second  with 
RETURN\\.  Moreover,  in  the  current  version  of  GUIDON  there  is  frequently  just  one 
rule  for  a  particular  task,  so  the  rule  is  simply  placed  "inline"  in  the  discourse 
procedure  that  invokes  it.  Figure  E.4  shows  the  second  step  of  the  procedure  for 
discussing  a  goal.  Notice  that  a  t-rule's  action  does  not  bind  local  variables,  so  a 
PROGN  is  used  (this  makes  it  possible  to  simply  use  RETURN\\  to  guarantee  return 
from  the  surrounding  discourse  procedure). 


tSTEPW  2 

(RULE  1.02  (IF  (KNOWN. BY. DEFN?  CURGOALPC) 

THEN  (PROGN  (SUBSTEP\\  1.  (STRATEGYW  DEFNRULF  RULEPK042 

CURGOALPC  DEFINITION. RULE)) 
(SUBSTEPW  11.  (DISCUSSW  RELATEDRULES . PROC0 1 1 

CURGOALPC  DEFINITION. RULE)) 
(SUBSTEPW  111.  (DISCUSSW  NOTDISCUBRULES . PROC007 

CURGOALPC ) ) 

(SUBSTEPW  1 v .  (ANSWERBLKINFO  NIL  CURGOALPC)) 
(SUBSTEPW  v.  (FORCE. UPDATE  CURGOALPC)) 

(SUB5TEPW  vl .  (RETURNW] 


Figure  E.4.  A  Single  T-rule  as  a  Procedure  Step 


Translation  of  Rules  and  Procedures 


GUIDON'S  rules  and  procedures  are  "translated"  from  INTERLISP  form  to  more  or 
less  readable  English  using  an  extension  of  the  techniques  developed  for  translating 
MYCIN's  rules.  A  special  procedure  (MACPROSE)  Is  used  to  format  translations  (print 
them  in  indented  form).  Forms  that  are  not  handled  by  this  procedure  are  converted 
to  a  lists  of  strings  by  the  MYCIN  function  PROSE.  Translation  mainly  consists  of 
recursively  substituting  translations  of  arguments  in  the  string  template  that 
describes  translation  of  a  function. 
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For  example,  the  code  (ZEROP  FACTORSTODO)  Is  translated  by  looking  up  the 
template  for  ZEROP,  ((1)  Is  "zero"),  and  substituting  the  translation  of  its  first 
argument  for  "(1)."  Templates  can  Include  arbitrary  LISP  code  that  computes  the 
appropriate  template  forms  or  output  strings,  e.g.,  to  translate  EQUAL  as  "equal  to" 
when  numbers  are  involved  and  "the  same  as"  otherwise.  In  addition,  calls  to 
STRATEGY\\  or  OISCUSS\\  are  printed  to  indicate  that  certain  flags  are  being  set. 
This  is  done  by  checking  to  see  which  actual  parameters  have  a  corresponding  formal 
parameter  with  the  FLAG  property.  1 


It  is  possible  to  randomly  access  any  t-rule  and  translate  it.  The  algorithm  is  as 
follows: 


1 )  Analyze  the  t-ruie  name  (T-RULExxx.nnn)  to  determine  the 
number  of  the  procedure  In  which  it  appears  (xxx);  nnn  is  the  index 
of  the  rule  in  that  procedure. 

't 

J 

2)  Analyze  the  list  of  discourse  procedure  names 
(<name>.DISCPROCxxx  or  <name>.RULEPACKETxxx)  to  find  which 
contains  the  t*rule. 


3)  Retrieve  the  filemap  of  the  disk  file  that  contains  this 
function  to  determine  the  byte  pointers  that  Indicate  where  the 
function  begins  and  ends. 


4)  Search  the  file  between  the  byte  pointer  locations  for  the 
string  "(RULE  xxx.nnn"  or  "[RULE  xxx.nnn"  and  return  the  LISP 
expression  at  this  point.  2 


6)  Use  MACPROSE  to  translate  the  record  (RULE  xxx.nnn  ...). 


Thus,  it  is  possible  to  index,  access,  and  annotate  t-rules  as  modular  units  (just 
like  MYCIN  rules),  while  they  are  organized  into  compiled  procedures  that  control  their 
evaluation. 


1  However,  when  the  argument  in  the  call  to  STRATEGY\\  or  DISCUSS\\  is  itself 
an  argument  to  the  function  that  contains  the  call,  it  is  not  possible  to  say  which  value 
ie  being  passed  on. 

2  This  code  was  written  by  Larry  Maslnter  and  Bill  van  Melle. 


368 


Appendix  F 

Discourse  Procedures  end  Tutoriei  Rules 


ago 


Discourse  Procedures  (PROP 


ADVICEAFTERHYP . PROC039  391 
ALLRULESFAILPARN.PROC035  391 
BlOCKDATA . PROC034  391 
COMPLETEOPARH.PROC005  392 
CONSULT. PROC003  393 
DEEPERSG .PROC019  393 
ENDOFRULE . PROC0 1 7  394 
FINISHGOAL . PROC006  395 
FREEGOAL . PROC002  395 
GIVENPARHOATA . PROC0 12  396 
GOAL . PROC001  398 
HELPFORGOAL .PROC016  402 
HELPFORHYP . PROC0 18  403 
HYPOTHESIS . PROC014  403 
NEVERASKEDPARM . PROC0 15  486 
NOTOISCUBRULES . PROC007  407 
ORIENTATION. PROC036  408 
PLEASE0I5CUSS . PROC022  408 
QUIZRULE .PROC021  489 
RECENTFAILS . PROC010  410 
RECENTSUCC . PROC043  410 
RELATEORULES . PROC0 1 1  411 
REMAININGTASK .PROC047  412 
RULE . PROC004  413 
SPECIFICHELP.PROC020  414 
TELLMEOPTION .PROC045  415 
UNKNOWNRULES . PROC046  416 


Rule  Pickets  (RULEPK) 


BACKGROUND. RULEPK048  416 
CHOSENRULEINTRO.RULEPK038  417 
DErNRULE .RULEPK042  417 
EXPERTDIDNTASK.RULEPK833  418 
FAILRULE .RULEPK023  419 
FOCUSRULES .RULEPK037  421 
GENOTYPE. RULEPK024  422 
HYPVALRESPONSE.RULEPK025  424 
IKNOWOPTION.RULEPK044  426 
NONEEOTOASK .RULEPK032  427 
PRESENTVALS .RULEPK026  428 
RULE  INTEREST. RULEPK027  428 
RULEUSED .RULEPK028  430 
RULEUSEDSREF.RULEPK029  432 
SAPPLIEDRULE . RULEPK04 1  433 
SUGGESTHYP .RULEPK030  434 
SUGGESTRULE . RULEPK03 1  435 
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ADVICE AF  TERHYP . PROC039 


Purpose:  Offer  a  summary  of  what  Is  known  so  far  and  the  opportunity  to 
terminate  discussion  of  the  hypothesis  topic 

StepI:  T-RULE39 .01 


If:  The  expert  has  not  applied  all  relevant  rules  for  the  hypothesis 
topic 

Then:  Offer  an  appraisal  of  what  more  needs  to  be  done  for  this  goal, 
suggesting  that  the  topic  be  terminated,  if  appropriate 
[Proc047] 

Else:  Substep  a.  Say:  oktoask 

Substep  b.  Supply  requested  data  to  the  student,  with 

introduction  and  related  Information,  as  appropriate 
[Proc012 ] 

->  The  request  for  data  has  already  been  determined 
to  be  appropriate 

Substep  c.  Record  that  discussion  of  the  hypothesis  topic  is 
complete 


ALLRULESFAILPARM . PROC035 


Purpose:  Discuss  the  fact  that  all  rules  fail  for  the  goal  currently  being 
discussed  and  consider  whether  It  is  appropriate  to  request  the 
reported  value 

Stepl:  Say:  no-successful -rules 
Step2:  T-RULE35.01 


If:  1)  The  goal  currently  being  discussed  has  not  been  discussed 
before, 

2)  The  student  has  not  requested  the  reported  value  already,  and 

3)  It  Is  possible  for  the  expert  to  request  the  value  of  the  goal 
currently  being  discussed 

Then:  Say:  norules-so-prompt 

Step3:  Present  all  relevant  case  data  for  this  goal  (the  ALLDATA  option) 
Step4 :  T-RULE35.02 


If:  It  Is  appropriate  now  to  request  the  reported  value  for  the  goal 
currently  being  discussed 
Then:  Substep  1.  Say:  case-data 

Substep  11.  Tell  the  student  the  reported  value  for  the  goal 
currently  being  discussed 


BLOCKOAT  A . PROCt  34 


Purpose:  Present  the  releted  factors  that  form  a  block  of  date 
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Stepl:  T-RULE34.01 


If:  Related  factors  that  form  a  block  of  data  has  not  been  discussed 
before 

Then:  Substep  1.  Give  a  fixed-text  definition  for  related  factors  that 
form  a  block  of  data 

Substep  11.  Record  that  related  factors  that  form  a  block  of 
data  has  been  mentioned  In  the  dialogue 

Step2:  Give  the  values  of  the  related  factors 

COMPLETEDPARM.PROC005 


Purpose:  Discuss  final  conclusion  for  the  goal  currently  being  discussed  by 
giving  the  final  answer  or  discussing  an  hypothesis 
Flagl:  The  student  has  requested  help  for  forming  an  hypothesis 
Flag2:  No  summary  of  evidence  should  be  offered  for  this  goal 
Flag3;  A  prompt  should  not  be  given  before  starting  discussion  of  this  goal 

Stepl:  T-RULE5.01 


If:  1)  There  are  rules  having  a  bearing  on  this  goal  that  have 
succeeded  and  have  not  been  discussed,  and 

2)  The  set  of  the  rules  having  a  bearing  on  the  goal  currently 
being  discussed  that  succeeded  has  one  member 
Then:  No  summary  of  evidence  should  be  offered  for  this  goal 

Step2:  Apply  the  first  rule  that  is  appropriate: 

T-RULE5 .02 


If:  1)  There  are  rules  having  a  bearing  on  this  goal  that  have 
succeeded  and  have  not  been  discussed, 

2)  The  set  of  rules  having  a  bearing  on  this  goal  that  have 
succeeded  and  have  not  been  discussed  has  one  member,  and 

3)  There  Is  strong  evidence  that  the  student  has  applied  the  rule 
Then:  Affirm  the  conclusions  made  by  the  rule  by  simply  stating  the 

key  factors  and  values  to  be  concluded 


T-RULE5.03 


If:  You  have  examined  rules  having  a  bearing  on  this  goal  that  have 
succeeded  and  have  not  been  discussed,  and  have  found  a  the  rule 
under  consideration  for  which  there  Is  not  strong  evidence  that 
the  student  has  applied  the  rule  under  consideration 
Then:  Substep  1.  If:  1)  A  prompt  should  be  given  before  starting 
discussion  of  this  goal,  and 

2)  The  student  has  not  requested  help  for  forming 
an  hypothesis 
Then:  Say:  hyp-ready 

Substep  11.  Discuss  the  student's  hypothesis  [Proc014] 

->  indicate  whether  The  student  has  requested  help 
for  forming  an  hypothesis 
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Step3:  T-RULE5 .04 


If:  1)  No  summary  of  evidence  should  be  offered  for  this  goal,  or 
2)  There  are  not  rules  having  a  bearing  on  this  goal  that  have 
succeeded 

Then:  Record  that  discussion  of  the  goal  currently  being  discussed  is 
complete 

Else:  Discuss  the  final  value  of  the  goal  currently  being  discussed 
and  wrap-up  the  discussion  before  returning  to  previous  topics 
[Proc006] 

->  A  remark  should  be  made  that  discussion  of  this  goal  Is  now 
complete 


CONSULT. PROC003 


Purpose:  Discuss  the  current  case 

Stepl:  Initialize  the  student  profile 

Step2:  ***m1ss1ng**e  case-selection 

Step3:  Read  In  the  appropriate  files  for  this  case 

Step4:  Print  out  Initial  data,  organized  by  objects  In  the  domain 

Step5:  Record  how  the  expert  used  this  introductory  Information 

Step6:  T-RULE3 .01 


If:  Discourse-mode  Is  goal -directed 
Then:  Substep  1.  Say:  goal-directed-intro 

Substep  11.  Discuss  the  goal  with  the  student  In  a  goal-directed 
mode  [Proc001] 

Else:  Substep  1.  Say:  freeform-intro 

Substep  11.  Discuss  the  goal  of  the  consultation  In  a  free-form 
mode  [Proc0i2] 

Step7 :  ***m1ss1ng*»*  prlntcrltlque 

Step8 :  ***m1ssing*e*  eval .userther 

StepD :  eeemlssingee*  special-remarks 

Stepie :  ***m1ssing*«*  laterstudy-info 

Stepl 1:  ***m1ss1ng*e«  fol low-upcases 

Stepl2 :  Save  the  student  model  and  discussion  record 

DEEPERSG . PROC019 


Purpose:  Present  the  path  from  the  goal  currently  being  discussed  to  the 
factor,  and  discuss  the  deepest  subgoal  for  which  the  i actor  is 
relevant 

Stepl:  T-RULE19.01 


If:  The  expert  has  reached  a  final  conclusion  about  the  new  subgoal 
Then:  Substep  1.  Say:  f inlsh-deeper-sg 

Substep  11.  Summarize  the  reasoning  path  from  the  factor  to  the 
goal  currently  being  discussed 

Substep  ill.  Discuss  final  conclusion  for  the  new  subgoal  by 

giving  the  final  answer  or  discussing  an  hypothesis 
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Substep 

Else:  Substep 
Substep 

Substep 

Substep 


[Proc805] 

1v.  Say  that  the  dialogue  is  returning  to  discussion  of 
the  goal  currently  being  discussed 
i.  Say:  discuss-deeper-sg 

11.  Summarize  the  reasoning  path  from  the  factor  to  the 
goal  currently  being  discussed 
ill.  Discuss  the  goal  with  the  student  in  a  goal- 
directed  mode  [Proc88l] 

1v.  Say  that  the  dialogue  is  returning  to  discussion  of 
the  goal  currently  being  discussed 


ENDOFRULE . PROC01 7 


Purpose:  Wrap  up  the  discussion  of  the  rule  currently  under  consideration 
Stepl :  T-RULE17.01 


If:  The  rule  currently  under  consideration,  is  the  goal  rule  of  the 
task 

Then:  Substep  1.  Say:  give-goalaction 

Substep  ii.  Record  that  the  rule  currently  under  consideration 
has  been  discussed 
Substep  ill.  Say:  end-rule 
Substep  iv.  Print  a  design  of  a  short  line 
Substep  v.  Exit  this  procedure 

Step2:  T-RULE17.02 


If:  It  is  not  true  that  the  student  model  is  strong  for  the  rule 
currently  under  consideration 
Then:  Substep  1.  Say:  mentionendof-rule 

Substep  11.  Record  that  the  rule  currently  under  consideration 
has  been  discussed 

Else:  Generate  a  question  about  the  goal  currently  being  discussed, 
using  the  'facts  of  the  rule’s  premise*  format  in  the  premise 
part  and  the  ’correct  conclusion*  format  in  the  conclusion 

Step3:  T-RULE17.03 


If:  This  Is  a  self-referencing  rule 
Then:  Say:  sref-concluslon 

Step4 :  T-RULE17 .04 


If:  This  is  not  the  first  rule  discussed  for  the  current  topic 
Then:  Say:  cum-evldence 

Step5:  T-RULE17.05 


If:  The  conclusion  made  by  this  rule  is  Insignificant 
Then:  Say:  insig-evld 

Step6:  Apply  the  first  rule  that  Is  appropriate: 
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T-RULE 1 7 . 86 


If:  The  expert  has  not  applied  all  relevant  rules  for  the  goal 
currently  being  discussed 
Then:  Say:  more-rules 


T-RULf 17.07 


If:  Discussion  of  the  goal  currently  being  discussed  is  not  complete 
Then:  Say:  not-disc 

Step7:  Discuss  rules  that  are  related  to  the  rule  which  was  Just  discussed 
[Proc011] 

Step8:  Say:  end-rule 

Step9:  Print  a  design  of  a  short  line 

F INISHGOAl . PROC006 


Purpose:  Discuss  the  final  value  of  the  goal  currently  being  discussed  and 
wrap-up  the  discussion  before  returning  to  previous  topics 
Flagl:  A  summary  should  be  given  of  this  evidence 

F1ag2:  A  remark  should  be  made  that  discussion  of  this  goal  is  now  complete 

Stepl:  Say:  printval 
StepZ :  T-RULE6.01 


If:  1)  Thera  are  rules  that  the  expert  used  to  determine  this  goal, 
and 

2)  A:  A  summary  should  be  given  of  this  evidence,  or 
B:  Ask:  want-f Inal-summary 

Then:  Present  the  conclusion  and  a  summary  of  evidence  that  supports 
it 

Step3:  *»*miss1np*a»  mentlon-doneparm- topics 
Step4 :  T-RULE6.92 


If:  There  are  rules  that  the  expert  used  to  determine  this  goal 
Then:  Print  a  design  of  asterisks 

Step5:  Record  that  discussion  of  the  goal  currently  being  discussed  is 
complete 

Step6:  T-RULE6.03 


If:  A  remark  should  be  made  that  discussion  of  this  goal  Is  now 
complete 

Then:  Say:  done-topic 


FREEGOAL . PROC002 


Purpose:  Discuss  the  goal  currently  being  discussed  in  a  free-form  mode 
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Stepl:  If:  The  student  has  requested  to  be  told  when  he  appears  to  be 
changing  the  subject 

Then:  ***miss1ng***  announce-goal-for-student 
Step2:  If:  The  expert  has  reached  a  final  conclusion  about  the  goal  currently 
being  discussed 
Then:  Exit  this  procedure 

Step3:  Accept  a  question,  hypothesis,  or  special  option  from  the  student 
Step4:  If:  The  student’s  current  Input  Is  a  hypothesis 

Then:  Substep  1.  If:  1)  The  hypothesis  topic  Is  not  a  subgoal  of  the 
goal  currently  being  discussed,  and 
Z)  The  student  has  requested  to  be  told  when  he 
appears  to  be  changing  the  subject 
Then:  ***m1ssing***  Irrelevant-hyp 
Substep  11.  Discuss  the  student’s  hypothesis  [Proc014] 

Else:  Substep  1.  Tell  the  student  the  reported  value  for  the  factor 
the  student  has  Inquired  about 

Substep  11.  If:  The  factor  the  student  has  Inquired  about  Is  not 
a  subgoal  of  (or  the  same  as)  the  goal  currently 
being  discussed 

Then:  ***m1ssing***  Irrelevant-ques 

Step5:  Go  to  stepZ 
GIVENPARMDATA . PROC012 


Purpose:  Supply  requested  data  to  the  student,  with  Introduction  and  related 
Information,  as  appropriate 

Flagl:  The  request  for  data  has  already  been  determined  to  be  appropriate 
Flag2:  The  student  has  requested  this  data  explicitly 

Stepl:  T-RULE12 .01 


If:  1)  The  student  has  not  requested  this  data  explicitly,  anu 
2)  The  factor  has  not  been  discussed  before 
Then:  Substep  1.  If:  This  factor  should  be  asked  about  before  an 
attempt  is  made  to  deduce  It  from  rules 
Then:  Say:  this  1$  a  'labdata'  parameter 
Else:  Say:  this  is  a  parameter  that  should  be  asked 
about  when  it  can’t  be  deduced  from  other 
evidence 

Substep  11.  Give  a  fixed-text  definition  for  the  factor 
Substep  111.  Record  that  the  factor  has  been  mentioned  In  the 
dialogue 

Else:  If:  The  student  has  not  requested  this  data  explicitly 
Then:  Say:  case-data 

5tep2:  Apply  the  first  rule  that  Is  appropriate: 

T-RULE12.02 


If:  The  request  for  data  has  already  been  determined  to  be  appropriate 
Then:  The  question  Is  valid 
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T-RULE12 .03 


If:  It  is  not  appropriate  now  to  request  the  reported  value  for  the 
factor 

Then:  Substep  i.  Inform  the  student  that  it  is  not  necessary  to 

request  the  reported  value  for  the  factor  in  this 
case.  [Rulepk032] 

Substep  ii.  Exit  this  procedure 


T-RULE12.04 


If:  The  factor  is  the  goal  currently  being  discussed 
Then:  The  question  is  valid 


T-RULE12 .05 


If:  1)  The  factor  is  not  a  subgoal  of  (or  the  same  as)  the  goal 
currently  being  discussed,  and 
2)  The  factor  Is  not  relevant  to  a  higher  subgoal 
Then:  Say:  irrelevant-question 


T-RULE12 .06 


If:  1)  The  factor  Is  relevant  to  a  higher  subgoal,  and 

2)  The  factor  is  not  a  subgoal  of  the  goal  currently  being 
discussed 

Then:  Say:  this  factor  Is  relevant  to  a  previous  goal,  but  not  the 
current  topic 


T-RULEI2.07 


If:  1)  There  are  not  related  factors  that  form  a  block  of  data,  and 
2)  There  are  not  rules  that  mention  the  factor  that  need  this 
information  In  order  to  be  applied 
Then:  Substep  a.  Say:  rules-evaled-already 

Substep  b.  If:  The  student  wants  details 

Then:  Describe  how  the  factor  is  relevant  to  the  goal 
currently  being  discussed  (the  USE  option) 
Substep  c.  Exit  this  procedure 
Else:  The  question  Is  valid 

Step3:  Tell  the  student  the  reported  value  for  the  factor 

Step4:  If  tills  factor  is  screened,  mark  the  screening  parameter  as  known; 

otherwise,  if  this  question  Is  a  screening  parameter,  give  the  value 
of  the  screened  factor 
Step5:  T-RULE12 .08 


If:  There  are  related  factors  that  form  a  block  of  data 

Then:  Present  the  related  factors  that  form  a  block  of  data  [ProcflVi  t 
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GOAL.PROC881 


Purpose:  Discuss  the  goal  with  the  student  in  a  goal-directed  mode 
Flagl:  A  prompt  should  not  be  given  before  starting  discussion  of  this  goal 


Stepl :  T-RULE1 .01 


If :  There  are 
this  case 
Then:  Substep 


Substep 

Substep 

Substep 


no  rules  for  concluding  about  this  goal  which  apply  to 

I.  If:  It  is  appropriate  now  to  request  the  reported 

value  for  the  goal  currently  being  discussed 
Then:  Supply  requested  data  to  the  student,  with 
introduction  and  related  information,  as 
appropriate  [Proc812] 

->  The  request  for  data  has  already  been 
determined  to  be  appropriate 

II.  Say:  no-rules-apply 

III.  Record  that  discussion  of  the  goal  currently  being 
discussed  Is  complete 

1v.  Exit  this  procedure 


Step? :  T-RULE1 .02 


If:  The  expert  deduced  the  goal  currently  being  discussed  from  a 
def ini t ion 

Then:  Substep  1.  Present  the  definition  rule  [Rulepk042] 

Substep  11.  Discuss  rules  that  are  related  to  the  rule  which  was 
just  discussed  [ProcBll] 

Substep  111.  Discuss  the  rules  having  a  bearing  on  the  goal 

currently  being  discussed  that  have  not  yet  been 
discussed  [Proc807] 

Substep  iv.  Give  the  values  of  the  related  factors 
Substep  v.  Record  that  discussion  of  the  goal  currently  being 
discussed  Is  complete 
Substep  vl.  Exit  this  procedure 

Step3:  T-RULE1 .83 


If:  All  of  the  rules  for  determining  the  goal  currently  being 
discussed  have  failed 

Then:  Substep  1.  Discuss  the  fact  that  all  rules  fall  for  the  goal 

currently  being  discussed  and  consider  whether  it  is 
appropriate  to  request  the  reported  value  [Proc835] 
Substep  11.  Record  that  discussion  of  the  goal  currently  being 
discussed  Is  complete 
Substep  111.  Exit  this  procedure 

•  Step4 :  T-RULE1 .84 


If:  l)  The  goal  currently  being  discussed  Is  the  goal  of  the 
consultation,  and 

2)  A:  The  student's  estimation  of  his  sophistication  (on  a  scale 
from  8  to  4)  Is  less  than  3,  or 

B:  The  goal  of  the  consultation  has  not  been  discussed  before 
Then:  Substep  1.  Say:  desc-goalrule 

Substep  11.  Show  a  sketch  of  the  subgoal  tree  for  the  goal 
currently  being  discussed 
Substep  111.  Say:  tell -opt ionname 
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Substep  1v.  Discuss  the  goal  rule  of  the  task  clause-by-clause 
[Proc804] 

Substep  v.  Say:  goodbye 

Substep  vi.  Record  that  the  goal  of  the  consultation  has  been 
mentioned  In  the  dialogue 
Substep  vii.  Exit  this  procedure 

Step5:  For  each  of  the  antecedent  rules  that  have  made  conclusions  about  the 
goal  currently  being  discussed 
Do:  T-RULE 1 .05 


If:  The  student  model  is  negative  for  the  rule  under  consideration 


Then:  Substep  1. 


Select  a  question  type  suitable  for  the  rule 
under  consideration  [Rulepk0?4] 


Substep  11.  Generate  a  question  based  upon  the  rule  under 
consideration  and  evaluate  the  student’s 
response 

Substep  ill.  Record  that  the  rule  under  consideration  has 
been  discussed 

Substep  iv.  Discuss  rules  that  are  related  to  the  rule  which 
was  just  discussed  [Proc011] 


5tep6:  T-RULEl. 86 


If:  It  is  appropriate  now  to  request  the  reported  value  for  the  goal 
currently  being  discussed 

Then:  Supply  requested  data  to  the  student,  with  introduction  and 
related  information,  as  appropriate  [Proc012] 

->  The  request  for  data  has  already  been  determined  to  be 
appropriate 

Step7 :  T-RULEl .07 


If:  1)  The  student  profile  Indicates  that  rule  conclusions  should  be 
mentioned,  and 

2)  There  are  rules  that  have  just  been  applied  by  the  expert  which 
pertain  to  the  goal  currently  being  discussed 
Then:  Substep  1.  Say:  completedrules 

Substep  11.  For  each  of  the  rules. to. mention 

Do:  Present  the  conclusion  and  a  summary  of  evidence 
that  supports  It 


StepB:  T-RULEl .08 


If:  Oiscusslon  of  the  goal  currently  being  discussed  is  complete 
Then:  Exit  this  procedure 

Step9:  T-RULEl. 09 


If:  The  expert  has  reached  a  final  conclusion  about  the  goal  currently 
being  discussed 

Then:  Substep  1.  Discuss  final  conclusion  for  the  goal  currently  being 
discussed  by  giving  the  final  answer  or  discussing  an 
hypothesis  [Proc805] 

Substep  11.  Exit  this  procedure 


400 


Stepl0 : 

T-RULE 1 . 10 


If:  1)  The  goal  currently  being  discussed  is  the  goal  of  the 
consultation,  and 

2)  The  expert  has  applied  all  relevant  rules  for  the  goal  of  the 
consultation 

Then:  Substep  i.  Wrap  up  the  discussion  of  the  goal  rule  of  the  task 
[Proc017] 

Substep  11.  Record  that  discussion  of  the  goal  currently  being 
discussed  is  complete 
Substep  111.  Exit  this  procedure 


Stepll : 

T-RULE 1. II 


If:  1)  You  have  not  asked  the  student  if  he  would  like  to  end 
discussion  of  the  current  topic, 

2)  Discussion  of  the  goal  currently  being  discussed  is  essentially 
complete,  and 

3)  Ask:  s-wants-hyp 

Then:  Substep  1.  Discuss  the  student's  hypothesis  [Proc014] 

Substep  ii.  If:  Discussion  of  the  goal  currently  being  discussed 
is  complete 

Then:  Substep  a.  Discuss  the  final  value  of  the  goal 
currently  being  discussed  and  wrap- 
up  the  discussion  before  returning 
to  previous  topics  [Proc006] 

•>  A  remark  should  be  made  that 
discussion  of  this  goal  is  now 
complete 

Substep  b.  Exit  this  procedure 

Stepl2 : 

T-RULE1 . 12 


If:  This  is  the  first  time  through  this  discussion  loop 
Then:  Substep  1.  Provide  initial  orientation  for  determining  the  goal 
currently  being  discussed  [Proc036] 

->  Indicate  whether  A  prompt  should  not  be  given 
before  starting  discussion  of  this  goal 
Substep  ii.  Define  this  is  the  first  time  through  this 
discussion  loop  to  be  nothing 

Stepl3: 

T-RULE1 . 13 


If:  There  are  rules  for  the  goal  currently  being  discussed  that  have 
recently  failed 

Then:  Discuss  Interesting  rules  which  are  known  to  fail  for  the  goal 
currently  being  discussed,  based  on  data  now  available  to  the 
student  [Proc010] 
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Stepl4: 


T-RULE1 . 14 


If:  There  are  rules  for  the  goal  currently  being  discussed  that  have 
recently  succeeded 

Then:  Discuss  interesting  rules  which  are  known  to  succeed  for  the 

goal  currently  being  discussed,  based  on  data  now  available  to 
the  student  [Proc843] 

Stepl5:  Accept  a  question,  hypothesis,  or  special  option  from  the  student 
Stepl6:  Process  the  student’s  input  as  follows: 

HYPOTHESIS:  Substep  i.  T-RULE1.15 


If:  The  hypothesis  topic  is  not  a  subgoal  of 
the  goal  currently  being  discussed 
Then:  Say:  irrelevant-hyp 

Substep  ii.  Discuss  the  student's  hypothesis  [Proc914] 
Substep  ill.  T-RULE1 . 16 

If:  The  expert  has  reached  a  final  conclusion 
about  the  hypothesis  topic 
Then:  Substep  a.  Discuss  the  final  value  of 
the  hypothesis  topic  and 
wrap-up  the  discussion 
before  returning  to  previous 
topics  [Proc8B6] 

->  A  remark  should  be  made 
that  discussion  of  this  goal 
is  now  complete 

Substep  b.  If:  The  hypothesis  topic  is 
the  goal  currently  being 
discussed 

Then:  Exit  this  procedure 
Else:  Say  that  the  dialogue 
is  returning  to 
discussion  of  the  goal 
currently  being 
discussed 

Substep  1v.  T-RULE1 . 17 

If:  l)  The  expert  has  not  reached  a  final 

conclusion  about  the  hypothesis  topic,  and 
2)  The  hypothesis  topic  is  not  the  goal 
currently  being  discussed 
Then:  Substep  a.  Discuss  the  goal  with  the 
student  In  a  goal-directed 
mode  [Proc001] 

Substep  b.  Say  that  the  dialogue  is 

returning  to  discussion  of 
the  goal  currently  being 
discussed 


DATA:  Substep  1.  Supply  requested  data  to  the  student,  with 

introduction  and  related  information,  as  appropriate 
[ProcBIZ] 

->  The  student  has  requested  this  data  explicitly 
Substep  11.  If:  The  question  is  not  valid 
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Then:  Go  to  stepl5 

Substep  111.  If:  The  factor  the  student  has  inquired  about  is 
not  a  factor  of  the  goal  currently  being 
discussed 

Then:  Present  the  path  from  the  goal  currently 

being  discussed  to  the  factoh  the  student  has 
inquired  about,  and  discuss  the  deepest 
subgoal  for  which  the  factor  the  student  has 
inquired  about  is  relevant  [Proc019] 
OTHERWISE:  Substep  i.  Invoke  the  special  option  procedure  for  the 

student's  input 

Substep  il.  If:  Nodatagivenflg 
Then:  Go  to  steplS 

Stepl7:  Go  to  step6 
HELPFORGOAL . PROC0 16 


Purpose:  Provide  help  for  determining  the  goal  currently  being  discussed 
Stepl:  T-RULE16 .01 


If:  1)  It  is  not  true  that  what  has  been  mentioned  in  the  recent 
context  of  the  Interaction,  and 

2)  The  student  has  not  seen  the  factors  for  this  goal 
Then:  Substep  i.  List  the  factors  that  occur  in  the  rules  that  have  a 
bearing  on  this  goal 
Substep  ii.  Say:  mention-optionname 
Substep  ill.  Exit  this  procedure 

Step? :  T-RULE16 .02 


If:  1)  There  are  not  successful  rules  for  determining  this  goal  that 
have  not  been  discussed,  or 

2)  There  are  self-referencing  rules  that  are  waiting  for  other 
rules  to  be  applied 

Then:  For  each  of  the  the  rules  having  a  bearing  on  the  goal  currently 
being  discussed  that  failed 
Do:  If:  The  rule  has  not  fired 

Then:  Present  the  rule  under  consideration  [Ru1epk023] 

->  Some  remark  should  be  made,  even  if  no 
presentation  is  possible 

Step3:  T-RULE16.03 


If:  There  are  not  successful  rules  for  determining  this  goal  that  have 
not  been  discussed 
Then:  Exit  this  procedure 

Step4:  To  determine  the  rules  under  contention,  compute  the  following: 

Select  the  rules  that  might  be  presented  to  the  student  from  among 
successful  rules  for  determining  this  goal  that  have  not  been 
discussed  [Ru1epk037] 

Step5:  To  determine  the  rule  being  considered,  compute  the  following:  for 

each  of  the  the  rules  under  contention,  determine  how  'interesting*  it 
is  to  present  the  rule  at  this  time,  returning  the  element  with  the 
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highest  score 

Step6:  Present  the  rule  being  considered  [Rulepk931] 

Step?T  Modify  the  student  knowledge  model  for  the  rule  being  considered  by 
-490 

HELPFORHYP . PROC0 18 


Purpose:  Select  a  rule  and  present  It  to  the  student  as  a  form  of  help  for 
reaching  an  hypothesis 

Stepl:  To  determine  the  rules  under  contention,  compute  the  following-. 

Select  the  rules  that  might  be  presented  to  the  student  from  among 
rules  having  a  bearing  on  this  goal  that  have  succeeded  and  have  not 
been  discussed  [Ru1epk037] 

Step?:  To  determine  the  rule  being  considered,  compute  the  following:  for 

each  of  the  the  rules  under  contention,  determine  how  'Interesting"  It 
Is  to  present  the  rule  at  this  time,  returning  the  element  with  the 
highest  score 

Step3 :  Suggest  a  rule  to  the  student  In  the  form  of  help  for  reaching  an 
hypothesis  [Rulepk939] 

HYPOTHESIS. PROC0 14 


Purpose:  Discuss  the  student's  hypothesis 

Flagl:  The  student  has  requested  help  for  forming  an  hypothesis 
Stepl:  T-RULE14 .01 


If:  It  Is  appropriate  now  to  request  the  reported  value  for  the 
hypothesis  topic 

Then:  Supply  requested  data  to  the  student,  with  Introduction  and 
related  Information,  as  appropriate  [Proc912] 

->  The  request  for  data  has  already  been  determined  to  be 
appropriate 

Step2:  Apply  the  first  rule  that  Is  appropriate: 

T-RULE14 .02 


If:  There  are  values  mentioned  by  the  student  when  he  asked  to  make  an 
hypothesis 

Then:  Define  values  hypothesized  by  the  student  to  be  values  mentioned 
by  the  student  when  he  asked  to  make  an  hypothesis 


T-RULE14 .03 


If:  The  student  has  not  requested  help  for  forming  an  hypothesis 
Then:  Ask  the  student  for  his  hypothesis  for  the  value  of  the 
hypothesis  topic 
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Step3:  T-RULE14 .04 


If:  1)  The  student  has  requested  help  for  forming  an  hypothesis,  and 
2)  There  are  rules  having  a  bearing  on  this  goal  that  have 
succeeded  and  have  not  been  discussed 
Then:  Substep  1.  Select  a  rule  and  present  it  to  the  student  as  a  form 
of  help  for  reaching  an  hypothesis  [Proc018] 

Substep  1i.  If:  There  are  rules  having  a  bearing  on  this  goal 

that  have  succeeded  and  have  not  been  discussed 
Then:  Substep  a.  Say:  hyp-ready 
Substep  b.  Go  to  step? 

Else:  Exit  this  procedure 

Else:  If:  1)  The  student  has  requested  help  for  forming  an  hypothesis, 
and 

2)  It  Is  not  true  that  Ask:  no-help 
Then:  Exit  this  procedure 

Step4:  Compare  the  student's  hypothesis  to  the  expert’s  current  hypothesis 
for  this  goal 

Step5:  Mark  the  rules  used  by  the  expert  to  reflect  their  consistency  with 
the  student's  hypothesis 

Step6:  For  each  of  the  rules  applied  by  the  expert 

Do:  Apply  modelling  t-rules  to  determine  the  belief  that  the  rule 
under  consideration  was  considered  by  the  student  [Rulepk828] 
5tep7:  For  each  of  the  rules  applied  by  the  expert 

Do:  In  view  of  the  model  for  all  other  rules,  apply  self-referencing 
t-rules  to  determine  whether  the  rule  under  consideration  was 
considered  by  the  student  [Rulepk829] 

Step8 :  T-RULE14 .05 


If:  There  are  correct  hypothesized  values  for  which  there  is  strong 
evidence  that  the  student  considered  the  appropriate  rules 
Then:  Say:  right-hyp 

Comment:  Strategy  for  strong  right  val  is  to  say  that  S  is  correct 
and  state  the  evidence. 

Step9:  Apply  the  first  rule  that  Is  appropriate: 

T-RULE14 .06 


If:  The  set  of  domain  rules  that  the  tutor  strongly  believes  were  used 
by  the  student  to  make  his  hypothesis  has  one  member 
Then:  Affirm  the  conclusions  made  by  the  rule  by  simply  stating  the 
key  factors  and  values  to  be  concluded 


T-RULE14 .07 


If:  There  are  domain  rules  that  the  tutor  strongly  believes  were  used 
by  the  student  to  make  his  hypothesis 
Then:  Substep  1.  Say:  strongrules-intro 

Substep  11.  Present  a  summary  of  evidence  that  the  hypothesis 

topic  is  correct  hypothesized  values  for  which  there 
v  is  strong  evidence  that  the  student  considered  the 
appropriate  rules 


SteplS: 
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T-RULE14.88 


If:  There  are  correct  hypothesized  values  for  which  the  student  may 
have  missed  some  relevant  evidence 
Then:  For  each  of  the  correct  hypothesized  values  for  which  the 
student  may  have  missed  some  relevant  evidence 
Do:  Respond  to  the  hypothesis  that  the  hypothesis  topic  is  this 
.  value  [Ru1epk825] 

Comment:  If  there  are  several  strong  rules  and  1  weak  one,  rule 

this  rulepk  will  state  the  strong  evidence  and  quiz  about 
the  weak  rule  (mixed  right  val  strategy). 


Stepll: 


T-RULE 14.89 


If:  There  are  correct  values  for  which  there  is  both  positive  and 
negative  evidence 

Then:  Generate  a  question  about  relevant  rules,  using  "fill-in  of 
known  factors"  format  in  the  premise  part  and  the  "correct 
conclusion"  format  in  the  conclusion 

Comment:  Strategy  for  ambig  right  val  is  to  see  If  he  knows  evidence 
contrary  to  his  hypothesis. 


Stepl2 : 


T-RULE 14.1 


If:  There  are  wrong  hypothesized  values 
Then:  For  each  of  the  wrong  hypothesized  values 

Do:  Respond  to  the  hypothesis  that  the  hypothesis  topic  is  this 
value  [Rulepk82$] 

Comment:  Rulepks  25  and  26  will  present  the  unknown  evidence 


Stepl3: 

T-RULE14.il 


If:  There  are  values  missing  from  the  student's  hypothesis 
Then:  Generate  a  question  about  relevant  rules,  using  "fill-in  of 
known  fact  rs*  format  in  the  premise  part  and  the  "correct 
conclusion"  format  In  the  conclusion 

Stepl4: 

T-RULE 14. 12 


If:  There  are  values  that  the  expert  explicitly  rejected 
Then:  Substep  1.  Say:  Intro-rejections 

Substep  11.  For  each  of  the  values  that  the  expert  explicitly 
rejected 

Do:  Present  the  conclusion  that  the  hypothesis  topic 
Is  (not)  this  value  [Rulepk826] 

Comment:  Strategy  for  rejected  vals  is  to  quiz  about  the  evidence 
that  the  expert  Is  considering 
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SteplS: 

T-RULE14 . 13 


If:  There  are  hypothesized  values  for  which  there  Is  only 
insignificant  evidence 

Then:  For  each  of  the  hypothesized  values  for  which  there  is  only 
Insignificant  evidence 
Do:  Substep  1.  Ask:  reauest-certainty 
Substep  11.  Say:  cf-reply 
Substep  ill.  Say:  insig-concls 
Substep  1v.  Present  a  summary  of  evidence  that  the 
hypothesis  topic  is  this  value 


Stepl6: 

T-RULE 14.14 


If:  1)  There  are  hypothesized  values  for  which  there  is  no  evidence  at 
all,  and 

2)  The  hypothesis  topic  is  not  numerically  valued 
Then:  Generate  a  question  about  the  hypothesis  topic,  using 

invalldfactors  format  In  the  premise  part  and  the  "correct 
conclusion"  format  in  the  conclusion 


Stepl7: 

T-RULE14 . 15 


If:  There  are  insignlf icmt  values  that  the  student  did  not  mention 
Then:  ***m1ss1ng««*  mention-insigvals 


Staple : 

T-RULE14.16 


If:  The  expert  has  not  reached  a  final  conclusion  about  the  hypothesis 
topic 

Then:  Offer  a  summary  of  what  Is  known  so  far  and  the  opportunity  to 
terminate  discussion  of  the  hypothesis  topic  [Proc039] 

Else:  Record  that  discussion  of  the  hypothesis  topic  is  complete 


NEVERASKEDPARH . PROC0 1 5 


Purpose:  Inform  the  student  that  the  expert  never  needed  to  ask  about  the 
factor  and  offer  details 

Stepl:  Say:  the  expert  never  needed  to  inquire  about  this  factor 
Step2 :  T-RULE15.81 


If:  There  are  rules  having  a  bearing  on  this  goal  that  have  succeeded 
Then:  Say:  rules-tellus 

Step3:  T-RULE 15. 82 


If:  1)  The  factor  Is  not  the  goal  currently  being  discussed, 

2)  There  are  rules  having  a  bearing  on  this  goal  that  have 
succeeded, 

3)  There  are  rules  having  a  bearing  on  this  goal  that  have  failed. 
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and 

4)  Ask:  wants-detalls 

Then:  Substep  1.  Discuss  the  goal  with  the  student  In  a  goal-directed 
mode  [Proc801] 

Substep  1i.  Say  that  the  dialogue  Is  returning  to  discussion  of 
the  goal  currently  being  discussed 


Step4:  T-RULE15.03 


If:  The  factor  Is  not  the  goal  currently  being  discussed 
Then:  Describe  how  the  factor  is  relevant  to  the  goal  currently  being 
discussed  (the  USE  option) 

Comment:  In  theory,  this  will  print  a  message  about  relevance  and  exit 


NOTD ISCUBRULES . PROC00  7 


Purpose:  Discuss  the  rules  having  a  bearing  on  the  goal  currently  being 
discussed  that  have  not  yet  been  discussed 

Stepl:  For  each  of  the  the  rules  having  a  bearing  on  the  goal  currently  being 
discussed  that  succeeded 
Do:  T-RULE7 .01 


If:  1)  The  rule  has  not  fired, 

2)  The  rule  has  not  been  discussed,  and 

3)  The  rule  under  consideration  is  on  the  lesson  plan 

Then:  Substep  1.  Hake  a  *>»*  design  with:  Considering  rules  that 
haven't  been  discussed 

Substep  11.  Oiscuss  the  rule  under  consideration  clause-by¬ 
clause  [Proc004) 

Step2:  For  each  of  the  the  rules  having  a  bearing  on  the  goal  currently  being 
discussed  that  failed 
Do:  T -RULE 7 .02 


If:  1)  The  rule  has  not  been  discussed,  and 
2)  A:  The  rule  has  not  fired,  or 

B:  The  rule  under  consideration  is  on  the  lesson  plan 
Then:  Substep  1.  Hake  a  ">»"  design  with:  Considering  rules  that 
haven’t  been  discussed 

Substep  11.  Present  the  rule  under  consideration  [Ru1epk023] 

Step3:  T-RULE7 .03 


If:  There  are  rules  not  used  by  the  expert  to  determine  this  goal 
Then:  •••missing***  unusedrules 

Step4 :  End  the  "»>•  design  with:  End  of  discussion  of  rules  for  this  factor 


ORIENTATION. PROC836 


Purpose:  Provide  Initial  orientation  for  determining  the  goal  currently 
being  discussed 

Flagl:  A  prompt  should  not  be  given  before  starting  discussion  of  this  goal 
Stepl:  T-RULE36 .01 


If:  A  prompt  should  be  given  before  starting  discussion  of  this  goal 
Then:  Say:  announce-goal-for-student 

Step? :  T-RULE36 .82 


If:  1)  The  number  of  rules  that  conclude  about  the  current  goal  is 
greater  than  5,  and 

2)  The  goal  currently  being  discussed  has  not  been  discussed 
before 

Then:  Substep  i.  Give  a  fixed-text  definition  for  the  goal  currently 
being  discussed 

Substep  ii.  Provide  orientation  for  finding  out  about  the  goal 
currently  being  discussed  by  listing  the  subgoals 
most  commonly  pursued  in  this  context 

Substep  iii.  Say:  orient-by-rule 

Substep  iv.  List  (typical)  values  for  the  goal  currently  being 
discussed 

Substep  v.  Record  that  the  goal  currently  being  discussed  has 
been  mentioned  in  the  dialogue 

Step3:  T-RULE36.03 


If:  Values  hypothesized  for  this  goal  which  have  already  been 
confirmed  in  discussion 
Then:  Say:  discussed-concl 


PLEASEDISCUSS.PROC022 


Purpose:  Change  the  dialogue  from  the  goal  currently  being  discussed  to  the 
new  topic,  as  the  student  has  requested 


Stepl:  T-RULE22 .81 


If:  Discussion  of  the  new  topic  is  complete 
Then:  Substep  1.  Say:  donetopic 

Substep  11.  Discuss  the  final  value  of  the  new  topic  and  wrap-up 
the  discussion  before  returning  to  previous  topics 
[Proc000] 

Substep  111.  Say  that  the  dialogue  is  returning  to  discussion  of 
the  goal  currently  being  discussed 
Substep  Iv.  Exit  this  procedure 

Step2 :  T-RULE22 .02 


If:  The  new  topic  Is  not  a  subgoal  of  (or  the  same  as)  the  goal 
currently  being  discussed 
Then:  Say:  will -proceed 
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Step3:  Discuss  the  goal  with  the  student  In  a  goal 'directed  mode  [Proc001] 
Step4:  Say  that  the  dialogue  Is  returning  to  discussion  of  the  goal  currently 
being  discussed 

QUIZRULE .PROC021 


Purpose:  Quiz  the  student  about  a  rule  for  determining  the  goal  currently 
being  discussed 

Flagl:  The  student  has  requested  a  hint 
Stepl :  T-RULE21 .81 


If:  The  student  has  requested  a  hint 

Then:  To  determine  the  rules  under  contention,  compute  the  following: 
the  rules  that  succeed  for  the  goal  currently  being  discussed 
that  have  not  been  discussed 

Else:  To  determine  the  rules  under  contention,  compute  the  following: 
the  subset  of  rules  which  are  relevant  for  the  goal  currently 
being  discussed  and  can  now  be  applied,  but  have  not  been 
discussed  for  which  the  rule  under  consideration  is  not  a  rule 
that  failed  inappropriately  for  quizzing  (i.e.,  it  has  factors 
whose  values  are  unknown  to  the  expert  or  has  a  tabular  factor 
with  a  null  value) 

Step2:  T-RULE21.82 


If:  There  are  not  the  rules  under  contention 
Then:  Substep  1.  Say:  no-quiz 

Substep  11.  Exit  this  procedure 

Step3:  To  determine  the  rule  being  considered,  compute  the  following:  for 

each  of  the  the  rules  under  contention,  determine  how  ‘interesting*  it 
is  to  present  the  rule  at  this  time,  returning  the  element  with  the 
highest  score 
Step4 :  T-RULE21 .83 


If:  1)  The  student  has  requested  a  hint,  and 

2)  A  hint  value  can  be  suggested  for  this  rule,  based  on  its 
effect  on  the  current  hypothesis 
Then:  Substep  1.  Say:  valu-hlnt 

Substep  11.  Define  the  rule  hinted  about  to  be  the  rule  being 
considered 

Substep  ill.  Define  what  has  been  mentioned  in  the  recent 

context  of  the  Interaction  to  be  a  hint  about  a 
rule 

Substep  iv.  Exit  this  procedure 
StepS:  T-RULE21 .04 


If:  The  student  has  requested  e  hint 
Then:  Substep  1.  Say:  no-hint 
Substep  11.  Go  to  stepl 

Step6:  Select  a  question  type  suitable  for  the  rule  being  considered 
[Rulepk024] 


Step7:  Generate  a  question  based  upon  the  rule  being  considered  and  evaluate 
the  student' s  response 

Step8:  Record  that  the  rule  being  considered  has  been  discussed 
RECENTFAILS . PROC0 10 


Purpose:  Discuss  Interesting  rules  which  are  known  to  fall  for  the  goal 
currently  being  discussed,  based  on  data  now  available  to  the 
student 

Stepl:  To  determine  the  rules  under  contention,  compute  the  following:  the 
subset  of  recentfalls  for  which 

1)  The  rule  has  not  been  discussed, 

2)  The  rule  under  consideration  Is  not  a  rule  that  failed 
Inappropriately  for  quizzing  (i.e.,  It  has  factors  whose  values  are 
unknown  to  the  expert  or  has  a  tabular  factor  with  a  null  value), 
and 

3)  The  rule  under  consideration  Is  on  the  lesson  plan 
Step2:  If:  There  are  not  the  rules  under  contention 

Then:  Exit  this  procedure 

Step3:  To  determine  the  rule  being  considered,  compute  the  following:  for 

each  of  the  the  rules  under  contention,  determine  how  "Interesting"  it 
Is  to  present  the  rule  at  this  time,  returning  the  element  with  u>f 
highest  score 
Step4:  T-RULE10.81 


If:  The  number  of  subgoals  remaining  to  be  determined  before  this  rule 
can  be  applied  Is  not  zero 
Then:  Exit  this  procedure 

Step5:  T-RULE10 .02 


If:  The  number  of  factors  appearing  In  this  rule  which  need  to  be 
asked  by  the  student  Is  not  zero 
Then:  Substep  1.  Say:  Intro-data 

Substep  11.  For  each  of  the  factors  whose  reported  value  must  be 
given  to  the  student  before  he  can  apply  the  rule 
under  consideration 

Do:  Tell  the  student  the  reported  value  for  the 
factor 

Stepb:  Select  a  question  type  suitable  for  the  rule  being  considered 
[Ru1epk024] 

Step7:  Generate  a  question  based  upon  the  rule  being  considered  and  evaluate 
the  student’s  response 

Step8:  Record  that  the  rule  being  considered  has  been  discussed 
RECENTSUCC . PROC043 


Purpose:  Discuss  Interesting  rules  which  are  known  to  succeed  for  the  goal 
currently  being  discussed,  based  on  data  now  available  to  the 
student 


Stepl:  For  each  of  the  rules  that  have  recently  succeeded 
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Do:  Apply  the  first  rule  that  Is  appropriate: 
T-RULE43.01 


If:  1)  The  rule  has  not  been  discussed, 

2)  There  Is  not  strong  evidence  that  the  student  has  applied 
the  rule  under  consideration,  and 

3)  There  Is  a  subgoal  that  the  student  probably  does  not  know 
Then:  Substep  1.  Print  a  design  of  a  line  of  dashes 

Substep  11.  Say:  Intro-recentsucc-tree 

Substep  111,  For  each  of  the  rules  In  this  subtree  that  the 
student  probably  cannot  apply  because  he  Is 
unable  to  determine  one  or  more  subgoals 
Oo:  Present  missing  rules  to  the  student  so 
that  he  can  determine  the  missing  subgoal, 
then  quiz  about  rules  that  use  these 
subgoals  [Proc046J 

Substep  1v.  Print  a  design  of  a  line  of  dashes 


T-RULE43.02 


If:  1)  The  rule  has  not  been  discussed, 

2)  There  Is  not  a  subgoal  that  the  student  probably  does  not 
know, 

3)  There  Is  not  strong  evidence  that  the  student  has  applied 
the  rule  under  consideration,  and 

4}  The  rule  under  consideration  Is  the  only  rule  for  the  goal 
currently  being  discussed  that  mentions  certain  case  data 
Then:  Substep  1.  Select  a  question  type  suitable  for  the  rule 
under  consideration  [Rulepk024] 

Substep  11.  Generate  a  question  based  upon  the  rule  under 
consideration  and  evaluate  the  student’s 
response 

Substep  111.  Record  that  the  rule  under  consideration  has 
been  discussed 


RELATEDRULES . PROC01 I 


Purpose:  Discuss  rules  that  are  related  to  the  rule  which  was  just  discussed 
Stepl :  T-RULE11 .01 

If:  The  goal  currently  being  discussed  Is  a  true/false  parameter 
Then:  Exit  this  procedure 

Comment:  Restrict  related  rule  quizzes  to  perms  with  real  values  so 
the  relation  Is  clear 


Step2:  For  each  of  the  randomly  reordered  set  of  rules  which  are  relevant  for 
the  goal  currently  being  discussed  and  can  now  be  applied,  but  have 
not  been  discussed 


412 


Do:  T-RULE11.02 


If:  1)  The  number  of  questions  asked  so  far  is  less  than  3, 

2)  It  is  not  true  that  the  premise  of  the  rule  under 
consideration  includes  an  iteration, 

3)  The  rule  under  consideration  is  not  a  rule  that  failed 
inappropriately  for  quizzing  (i.e.,  it  has  factors  whose 
values  are  unknown  to  the  expert  or  has  a  tabular  factor  with 
a  null  value),  and 

4)  The  rule  under  consideration  does  conclude  about  some  of 
the  same  values  as  the  rule  just  discussed 

Then:  Substep  1.  Make  a  *>>>*  design  with:  Considering  related 
rules 

Substep  ii.  Select  a  question  type  suitable  for  the  rule 
under  consideration  [Rulepk024] 

Substep  111.  Generate  a  question  based  upon  the  rule  under 
consideration  and  evaluate  the  student's 
response 

Substep  iv.  Record  that  the  rule  under  consideration  has 
been  discussed 

Step3:  End  the  “>>>"  design  with:  End  of  our  discussion  of  related  rules 
REMAININGTASK . PROC04  7 


Purpose:  Offer  an  appraisal  of  what  more  needs  to  be  done  for  this  goal, 
suggesting  that  the  topic  be  terminated,  if  appropriate. 

Stepl:  Compare  the  student's  hypothesis  to  the  expert's  current  hypothesis 
for  this  goal 
Step2:  T-RULE47.01 


If:  There  are  values  missing  from  the  student’s  hypothesis 
Then:  Say:  misslngf inalvals 
Else:  Say:  hyparmrulesnotdone 

'  Step3:  T-RULE47 .02 


If:  1)  There  are  not  values  missing  from  the  student's  hypothesis,  and 
2)  Discussion  of  the  hypothesis  topic  is  essentially  complete 
Then:  Say:  goal -nearly-done 

Step4:  T-RULE47 .03 


If:  Ask:  want-shortcut 

Then:  Substep  a.  If:  1)  The  expert  did  find  it  necessary  to  inquire 
about  the  hypothesis  topic,  and 
2)  The  student  has  not  requested  the  reported 
value  already 

Then:  Supply  requested  data  to  the  student,  with 
Introduction  and  related  information,  as 
appropriate  [Proc012] 

->  The  request  for  data  has  already  been 
determined  to  be  appropriate 

Substep  b.  Present  all  relevant  case  data  for  this  goal  (the 
ALLDATA  option) 


i 
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Substep  c.  Record  that  discussion  of  the  hypothesis  topic  is 
complete 

Else:  Tf:  1)  There  is  a  curval,  and 

2)  Ask:  s-wants-hyp 

Then:  Present  the  conclusion  and  a  summary  of  evidence  that 
supports  It 


RULE .PROC004 


Purpose:  Discuss  the  rule  currently  under  consideration  clause-by-clause 


Stepl:  Print  a  design  of  a  short  line 
Step2:  Say:  announce-rule 
5tep3 :  T-RULE4.01 


If:  1)  The  rule  currently  under  consideration  is  not  the  goal  rule  of 
the  task, 

2)  It  is  not  true  that  the  premise  of  the  rule  currently  under 
consideration  includes  an  iteration, 

3)  The  student  model  is  uncertain  for  the  rule  currently  under 
consideration,  and 

4)  The  rule  currently  under  consideration  is  the  only  rule  that 
concludes  this  value  for  the  goal  currently  being  discussed 

Then:  Generate  a  question  about  the  goal  currently  being  discussed, 
using  newsubgoals  format  in  the  premise  part  and  the  "correct 
conclusion*  format  in  the  conclusion 


Step4:  If:  There  is  a  factor  that  remains  to  be  considered  before  this  rule 
can  be  applied 

Then:  Substep  i.  If:  A  new  clause  in  the  rule  will  now  be  considered 

Then:  Substep  a.  If:  1)  The  number  of  factors  in  this 
clause  is  equal  to  1,  and 
2)  The  factor  under 
consideration  is  the  goal 
currently  being  discussed 
Then:  Go  to  step4 

Substep  b.  If:  All  factors  in  this  clause  have 
not  been  discussed 
Then:  Say:  mustdoclause 
Else:  Substep  i.  Say:  al Idoneclause 
Substep  ii.  Go  to  step4 

Substep  11.  If:  1)  There  are  no  rules  for  concluding  about  this 
goal  which  apply  to  this  case,  or 
2)  Discussion  of  the  factor  under  consideration 
is  not  complete 

Then:  Substep  1.  If:  It  is  not  true  that  a  new 

clause  in  the  rule  will  now  be 
considered 

Then:  Say:  mustdo-factor 
Substep  ii.  Discuss  the  goal  with  the  student 
in  a  goal-directed  mode  [Proc001] 
Substep  ill.  Say:  rule-return 
Substep  111.  Go  to  step4 

StepS:  Wrap  up  the  discussion  of  the  rule  currently  under  consideration 
[Proc017] 


414 


SPECIF ICHELP . PROC020 


Purpose:  Provide  assistance  for  using  the  subtopic  mentioned  In  the  reouest 
for  help  to  determine  the  goal  currently  being  discussed 

Stepl :  T-RULE20 .01 


If:  It  Is  appropriate  now  to  request  the  reported  value  for  the 
subtopic  mentioned  In  the  request  for  help 
Then:  Supply  requested  data  to  the  student,  with  Introduction  and 
related  Information,  as  appropriate  [Proc012] 

->  The  request  for  data  has  already  been  determined  to  be 
appropriate 

Step2 :  T-RULE20 .02 


If:  The  subtopic  mentioned  In  the  request  for  help  Is  a  subgoal  of  (or 
the  same  as)  the  goal  currently  being  discussed 
Then:  Describe  how  the  subtopic  mentioned  In  the  request  for  help  Is 
relevant  to  the  goal  currently  being  discussed  (the  USE  option) 
Else:  Say:  Irrelevant-ques 

5tep3:  T-RULE20.03 


If:  1)  The  expert  has  reached  a  final  conclusion  about  the  subtopic 
mentioned  in  the  request  for  help,  and 

2)  Discussion  of  the  subtopic  mentioned  In  the  request  for  help  Is 
complete 

Then:  Exit  this  procedure 
Step4 :  T-RULE20 .04 


If:  1)  The  expert  has  not  reached  a  final  conclusion  about  the 
subtopic  mentioned  In  the  request  for  help,  and 
2)  Discussion  of  the  subtopic  mentioned  In  the  request  for  help  Is 
not  complete 

Then:  Substep  1.  Say:  dlscuss-help-toplc 

Substep  11.  Discuss  the  goal  with  the  student  In  a  goal-directed 
mode  [Proc001] 

StepS:  T-RULE20 .05 


If:  Discussion  of  the  subtopic  mentioned  In  the  request  for  help  Is 
not  complete 

Then:  Substep  1.  Say:  Intro-hyp 

Substep  11.  Discuss  final  conclusion  for  the  subtopic  mentioned 
In  the  request  for  help  by  giving  the  final  answer 
or  discussing  an  hypothesis  [Proc005] 

->  The  student  has  requested  help  for  forming  an 
hypothesis 

StepS:  Say  that  the  dialogue  Is  returning  to  discussion  of  the  goal  currently 
being  discussed 
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TELLNEOPTIO# . PROCf 45 

Purpose:  Terminate  discussion  of  this  goal,  as  tha  student  has  requested 
Flagl:  A  summary  should  be  given  of  this  evidence 

Stepl:  T-RULE45.81 

If:  Discussion  of  this  goal  Is  complete 
Then:  Substep  1.  Say:  glve-val 

Substep  11.  Exit  this  procedure 

Step2:  Apply  the  first  rule  that  Is  appropriate: 

T-RULE45.02 


If:  All  of  the  rules  for  determining  this  goal  have  failed 
Then:  Discuss  the  fact  that  all  rules  fall  for  this  goal  and  consider 
whether  It  Is  appropriate  to  request  the  reported  value 
[Proc035] 


Apply  all  rules  that  are  appropriate: 
T-RULE45 .83 


If:  1)  The  expert  did  find  It  necessary  to  Inquire  about  this  goal, 
and 

2)  The  student  has  not  requested  the  reported  value  already 
Then:  Supply  requested  data  to  the  student,  with  Introduction  and 
related  information,  as  appropriate  [Proc012] 

->  The  request  for  date  has  already  been  determined  to  be 
appropriate 


T*RULE45.04 


If:  There  are  rules  that  the  expert  used  to  determine  this  goal 
Then:  Discuss  the  final  value  of  this  goal  and  wrap-up  the  discussion 
before  returning  to  previous  topics  [Proe006] 

•>  Indicate  whether  A  summary  should  be  given  of  this  evidence 

Step3 :  T-RULE45.05 


If:  1)  The  goal  currently  being  discussed  Is  not  this  goal,  and 
2)  There  are  rules  that  the  expert  used  to  determine  this  goal 
Then:  Substep  1.  Say:  topic-done 

Substep  11.  Say  that  the  dialogue  Is  returning  to  discussion  of 
the  goal  currently  being  discussed 

Step4:  Record  that  discussion  of  this  goal  Is  comploto 


3. 
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UNKNOVNRULES . PROC046 


Purpose:  Present  missing  rules  to  the  student  so  that  he  can  determine  the 
missing  subgoal,  then  quiz  about  rules  that  use  these  subgoals 

Stepl;  Say:  Intro-newvals 

Step?:  Affirm  the  conclusions  made  by  rules  probably  not  known  by  the  student 
by  simply  stating  the  key  factors  and  values  to  be  concluded 
Step3:  Discuss  final  conclusion  for  the  subgoal  that  these  rules  conclude 

about  by  giving  the  final  answer  or  discussing  an  hypothesis  [Proc005] 
->  No  summary  of  evidence  should  be  offered  for  this  goal 
Step4:  T-RULE46 .01 


If:  There  are  other  unknown  subgoals  that  are  used  In  conjunction  with 
this  subgoal  that  the  student  does  not  know 
Then:  Exit  this  procedure 

Step5:  Select  a  question  type  suitable  for  the  rule  that  uses  this  subgoal 
[Rulepk024] 

Step6:  Generate  a  question  based  upon  the  rule  that  uses  this  subgoal  and 
evaluate  the  student's  response 

Step7:  Record  that  the  rule  that  uses  this  subgoal  has  been  discussed 
BACKGROUND. RULEPK040 


Purpose:  Determine  the  a  priori  belief  that  the  student  knows  the  rule 
Stepl:  Apply  all  rules  that  are  appropriate: 

T-RULE40.01 


If:  The  sophistication  level  of  the  rule  Is  not  greater  than  the 
student's  estimation  of  his  sophistication  (on  a  scale  from  0  to 
4) 

Then:  Modify  the  cumulative  'background  component*  of  the  student 
knowledge  model  by  If  the  sophistication  level  of  the  rule 
minus  the  student's  estimation  of  his  sophistication  (on  a  scale 
from  0  to  4)  Is  zero  then  400,  else  800 


T-RULE40.0Z 


If:  1)  The  sophistication  level  of  the  rule  Is  not  greater  than  the 
student’s  estimation  of  his  sophistication  (on  a  scale  from  0  to 
4), 

2)  The  rule  Is  definitional,  and 

3)  It  Is  not  true  that  the  rule  mentions  tables  or  lists  in  its 
premise 

Then:  Modify  the  cumulative  'background  component*  of  the  student 
knowledge  model  by  .9 


T-RULE40.03 


If:  The  premise  of  the  rule  Includes  an  Iteration 

Then:  Modify  the  cumulative  'background  component*  of  the  student 


i 


i 
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knowledge  model  by  -.9 

Step2:  Modify  the  student  knowledge  model  for  the  rule  by  the  cumulative 
"background  component*  of  the  student  knowledge  model 
Step3:  Exit  this  procedure,  returning  as  a  result:  the  cumulative  "background 
component"  of  the  student  knowledge  model 

CHOSENRULEINTRO . RULE  PK038 


Purpose:  Select  an  Introduction  for  beginning  discussion  of  the  rule  being 
considered 

Stepl:  Apply  the  first  rule  that  Is  appropriate: 

T-RULE38.01 


If:  1)  What  has  been  mentioned  In  the  recent  context  of  the 

Interaction  Is  either  a  deeper  subgoal  or  e  subtopic  relevant  to 
the  current  goal,  and 
2 )  There  is  a  factor  being  focussed  upon 
Then:  Say:  focus-intro 


T-RULE38.02 


If:  A  hint  value  can  be  suggested  for  this  rule,  based  on  Its  effect 
on  the  current  hypothesis 
Then:  Say:  valu-lntro 


T-RULE38.I3 


If:  True 

Then:  Say:  default-rule-intro 


DEFNRULE .RULEPK042 


Purpose:  Present  the  definition  rule 
Stepl:  Apply  the  first  rule  that  Is  appropriate: 
T-RULE42.01 


If:  1)  The  rule  has  fired, 

2)  The  student  model  is  strong  for  the  definition  rule,  or 

3)  The  definition  rule  Is  one  of  self-referencing  rules  that  are 
waiting  for  other  rules  to  be  applied 

Then:  Present  the  conclusion  about  the  goal  currently  being  discussed 
made  by  the  definition  rule  [Rulepk826] 


0 
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T-RULE42.02 


If:  1)  The  number  of  subgoals  remaining  to  be  determined  before  this 
rule  can  be  applied  is  zero, 

2)  This  topic  Is  not  a  true/false  parameter,  and 

3)  The  definition  rule  is  the  only  rule  that  concludes  this  value 
for  the  goal  currently  being  discussed 

Then:  Generate  a  question  about  the  goal  currently  being  discussed, 

using  "given  new  facts*  format  in  the  premise  part  and  "fill-in" 
format  in  the  conclusion 


T-RULE42 .03 


If:  The  number  of  subgoals  remaining  to  be  determined  before  this  rule 
can  be  applied  Is  zero 

Then:  Generate  a  question  about  the  goal  currently  being  discussed, 

using  "fill-in  undiscussed  facts*  format  in  the  premise  part  and 
the  "correct  conclusion*  format  in  the  conclusion 


T-RULE42 .04 


If:  True 

Then:  Substep  1.  Say:  Intro-defn-rule 

Substep  11.  Discuss  the  definition  rule  clause-by-clause 
[Proc004] 


EXPERTDIDNTASK .RULEPK033 


Purpose:  Explain  why  the  expert  did  not  ask  about  the  factor 
Stepl:  Apply  the  first  rule  that  Is  appropriate: 

T-RULE33.01 


If:  There  Is  a  definitional  rule  that  states  that  this  factor  should 
not  be  asked  for  this  case 
Then:  Substep  i.  Say:  dontask-rule 

Substep  11.  Record  that  the  definition  rule  has  been  discussed 


T-RULE33.02 


If:  The  expert  deduced  the  factor  from  a  definition 
Then:  Present  the  definition  rule  [Rulepk042] 


T-RULE33.03 


If:  A  screening  question  is  asked  before  considering  the  factor 
Then:  Substep  1.  Define  the  fector  the  student  has  Inquired  about  to 
be  the  screening  parameter 
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Substep  11.  Supply  requested  data  to  the  student,  with 

Introduction  and  related  Inforiaatlon,  as  appropriate 
[Proc012] 

->  The  student  has  requested  this  data  explicitly 
Comment:  See  If  expert  asked  the  screen  parm 


T-RULE33.04 


If:  True 

Then:  Inform  the  student  that  the  expert  never  needed  to  ask  about  the 
factor  and  offer  details  [Proc015J 


FAILRULE .RULEPK023 


Purpose:  Present  the  rule  currently  under  consideration 

Flag!:  Some  remark  should  be  made,  even  If  no  presentation  Is  possible 

Stepl:  Apply  the  first  rule  that  Is  appropriate: 

T-RULE23.01 


If:  1)  The  rule  has  fired,  and 

2)  The  premise  of  the  rule  currently  under  consideration  Includes 
an  Iteration 

Then:  Generate  a  question  about  the  goal  currently  being  discussed, 
using  the  "facts  of  the  rule's  premise"  format  In  the  premise 
part  and  the  "correct  conclusion*  format  In  the  conclusion 


T-RULE23 .02 


If:  1)  The  rule  has  fired,  and 

2)  This  Is  not  a  rule  that  failed  and  has  factors  whose  values  are 
unknown  to  the  expert  or  a  tabular  factor  with  a  null  value 
Then:  Substep  1.  Select  a  question  type  suitable  for  the  rule 
currently  under  consideration  [Rulepk024] 

Substep  11.  Generate  a  question  based  upon  the  rule  currently 
under  consideration  and  evaluate  the  student's 
response 


T-RULE23.03 


If:  1)  The  number  of  subgoals  remaining  to  be  determined  before  this 
rule  can  be  applied  Is  zero,  and 

2)  A:  The  premise  of  the  rule  currently  under  consideration 
Includes  an  Iteration,  or 

B:  This  Is  a  rule  that  failed  and  has  factors  whose  values  are 
unknown  to  the  expert  or  a  tabular  factor  with  a  null  value 
Then:  Say:  complex-fall 
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T-RULE23.04 


If:  1)  The  number  of  subgoals  remaining  to  be  determined  before  this 
rule  can  be  applied  Is  zero, 

2)  The  number  of  factors  appearing  in  this  rule  which  need  to  be 
asked  by  the  student  is  equal  to  1,  and 

3)  A  value  Is  not  known  for  factors  whose  reported  value  must  be 
given  to  the  student  before  he  can  apply  the  rule  under 
consideration 

Then:  Substep  1.  Say:  intro-factor 

Substep  il.  Tell  the  student  the  reported  value  for  the  factor 
Substep  111.  If:  The  factor  is  a  true/false  parameter 

Then:  Generate  a  question  about  the  goal  currently 
being  discussed,  using  “right  premise 
hypothesis*  format  in  the  premise  part  and 
the  “correct  conclusion"  format  in  the 
conclusion 

Else:  Generate  a  question  about  the  goal  currently 
being  discussed,  using  “multiple  choice" 
format  in  the  premise  part  and  the  “correct 
conclusion"  format  in  the  conclusion 


T-RULE23.05 


If: 

Then: 


The  number  of  subgoals  remaining  to  be  determined  before  this  rule 
can  be  applied  is  zero 


Substep  i.  Say:  intro-newdata 

Substep  11.  For  each  of  the  factors  whose  reported  value  must 
given  to  the  student  before  he  can  apply  the  rule 
under  consideration 

Do:  Tell  the  student  the  reported  value  for  the 
factor 

Substep  111.  Select  a  question  type  suitable  for  the  rule 
currently  under  consideration  [Rulepk024] 

Substep  1v.  Generate  a  question  based  upon  the  rule  currently 
under  consideration  and  evaluate  the  student's 
response 


be 


T-RULE23.G6 


If:  There  Is  a  subgoal  appearing  in  this  rule  which  must  be  determined 
for  one  or  more  rules  that  succeed 
Then:  Substep  1.  Say:  consider-newgoal 

Substep  11.  Discuss  the  goal  with  the  student  in  a  goal-direct''* 
mode  [Proc801] 

Substep  111.  Say  that  the  dialogue  Is  returning  to  discussion  of 
the  goal  currently  being  discussed 


T-RULE23.D7 


If:  1)  The  number  of  subgoals  remaining  to  be  determined  before  this 
rule  can  be  applied  Is  zero, 

2)  The  number  of  factors  appearing  In  this  rule  which  need  to  be 


» 


421 


asked  by  the  student  Is  not  zero,  and 

3)  This  Is  not  a  rule  that  failed  and  has  factors  whose  values  are 
unknown  to  the  expert  or  a  tabular  factor  with  a  null  value 
Then:  Substep  1.  For  each  of  the  factors  whose  reported  value  must  be 
given  to  the  student  before  he  can  apply  the  rule 
under  consideration 

Do:  Tell  the  student  the  reported  value  for  the 
factor 

Substep  11.  Say:  motlv-toplc 


T-RUIE23 .08 


If:  1)  This  Is  not  a  rule  that  failed  and  has  factors  whose  values  are 
unknown  to  the  expert  or  a  tabular  factor  with  a  null  value, 

2)  The  number  of  subgoals  remaining  to  be  determined  before  this 
rule  can  be  applied  Is  equal  to  1,  and 

3)  The  number  of  factors  appearing  In  this  rule  which  need  to  be 
asked  by  the  student  Is  zero 

Then:  Substep  1.  Say:  newgoal-motlv 

Substep  11.  Discuss  the  goal  with  the  student  In  a  goal-directed 
mode  [Proc081] 

Substep  111.  Say:  fall-reminder 

Substep  1v.  Say  that  the  dialogue  Is  returning  to  discussion  of 
the  goal  currently  being  discussed 


T-RULE23.09 


If:  The  rule  has  not  fired 

Then:  Substap  1.  Force  this  rule  to  fire  without  discussing  It 
Substep  11.  If:  Some  remark  should  be  made,  even  If  no 
presentation  Is  possible 
Then:  Say:  no-qulz-posslble 

5tep2:  Record  that  the  rule  currently  under  consideration  has  been  discussed 
FOCUSRULES.RULEPK037 


Purpose:  Select  the  rules  that  might  be  presented  to  the  student  from  amonq 
successful  rules  for  determining  this  goal  that  have  not  been 
discussed 

Stepl :  Apply  the  first  rule  that  Is  appropriate: 

T-RULE37.01 


If:  1)  What  has  been  mentioned  In  the  recent  context  of  the 
Interaction  Is  a  hint  about  a  rule,  and 
2)  The  rule  hinted  about  Is  one  of  successful  rules  for 
determining  this  goal  that  have  not  been  discussed 
Then:  Exit  this  procedure,  returning  as  a  result:  the  rule  hinted 
about 


I 
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T-RULE37.0Z 


If:  1)  What  has  been  mentioned  In  the  recent  context  of  the 

Interaction  Is  a  suggestion  to  pursue  a  particular  hypothesis,  and 
2)  To  determine  the  rules  under  contention,  compute  the  following: 
the  Intersection  of  the  set  of  rules  Implicitly  referred  to  when 
the  tutor  suggested  that  the  student  pursue  a  particular 
hypothesis  and  the  set  of  successful  rules  for  determining  this 
goal  that  have  not  been  discussed 
Then:  Exit  this  procedure,  returning  as  a  result:  the  rules  under 
contention 


T-RUIE37.03 


If:  1)  What  has  been  mentioned  In  the  recent  context  of  the 

Interaction  Is  either  a  deeper  subgoal  or  a  subtopic  relevant  to 
the  current  goal,  and 

2)  To  determine  the  rules  under  contention,  compute  the  following: 
those  successful  rules  for  determining  this  goal  that  have  not 
been  discussed  that  mention  the  factor  being  focussed  upon  In 
their  premise 

Then:  Exit  this  procedure,  returning  as  a  result:  the  rules  under 
contention 


T-RULE37 .04 


If:  True 

Then:  Substep  1.  Define  what  has  been  mentioned  In  the  recent  context 
of  the  interaction  to  be  nothing 
Substep  11.  Exit  this  procedure,  returning  as  a  result: 

successful  rules  for  determining  this  goal  that  have 
not  been  discussed 


GENOTYPE. RULEPK024 


Purpose:  Select  a  question  type  suitable  for  the  rule  under  discussion 
Stepl:  Apply  all  rules  that  are  appropriate: 

T-RUIE24.81 


If:  True 

Then:  Define  the  valid  conclusion  formats  for  a  question  to  be  the 
"correct  conclusion" 


T-RULE24.02 


If:  I)  The  rule  being  presented  does  succeed,  or 
2)  This  topic  Is  not  a  true/false  parameter 
Then:  Include  the  "wrong  conclusion"  In  the  list  of  the  valid 
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conclusion  formats  for  a  question 

Comment:  If  yes/no  rule  failed,  there  is  the  danger  that  it  will  be 

stated  as  another,  correct  rule  --  e.g.,  when  X  =>  Y  and  not 
X  ®>  not  Y  ere  valid  rules 


T-RULE24.03 


If:  1)  This  rule  does  not  mention  all  valid  hypotheses  In  Its 
conclusion,  and 

2)  This  topic  Is  not  a  true/false  parameter 
Then:  Include  "multiple  choice*  in  the  list  of  the  valid  conclusion 
formats  for  a  question 

Comment:  Can't  make  multiple  choice  when  all  possible  vals  are  in  the 
conclusion 


T-RULE24 .04 


If:  1)  This  Is  a  quiz  about  two  related  rules, 

2)  There  are  related  hypotheses  in  the  two  rules,  and 

3)  There  are  not  contrary  hypotheses  made  by  the  two  related  rules 
Then:  Include  "confirmation  of  related  evidence"  in  the  list  of  the 

valid  conclusion  formats  for  a  question 

Comment:  Confirmation  question  requires  that  there  be  a  second  rule 
with  some  of  the  same  values  and  same  sign  cfs,  but  no 
contrary  evidence 


T-RULE24 .05 


If:  True 

Then:  To  determine  the  format  for  the  conclusion  part  of  the  question, 
compute  the  following:  a  random  selection  from  among  the  valid 
conclusion  formats  for  a  question 

Step2:  Apply  all  rules  that  are  appropriate: 

T-RULE24.06 


If:  1)  The  format  for  the  conclusion  part  of  the  question  Is  not 
either  "multiple  choice"  or  the  "wrong  conclusion",  or 
2)  The  rule  being  presented  does  succeed 
Then:  Include  the  "facts  of  the  rule's  premise"  In  the  list  of  the 
valid  premise  formats  for  a  question 

Comment:  If  rule  falls  can't  give  facts  and  a  wrong  conclusion  or 
multiple  choice 
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T-RULE24.07 


If:  1)  The  format  for  the  conclusion  part  of  the  question  is  not 
either  "multiple  choice"  or  the  "wrong  conclusion",  and 
2)  There  are  factors  In  the  rule's  premise  that  are  not  true/false 
parameters 

Then:  Include  "multiple  choice"  in  the  list  of  the  valid  premise 
formats  for  a  question 

Comment:  Use  multiple  choice  premise  when  action  is  not  multi  or  wrong 
and  there  is  a  non-yes/no  premise  factor 


T-RULE24 .08 


If:  The  rule  being  presented  does  not  succeed 

Then:  Include  "right  premise  hypothesis"  in  the  list  of  the  valid 
premise  formats  for  a  question 

Comment:  Use  right  hypothesis  only  when  rule  fails 


T-RULE24 .09 


If:  1)  There  are  factors  in  the  rule’s  premise  that  are  not  true/false 
parameters,  and 

2)  A:  The  format  for  the  conclusion  part  of  the  question  is  not 
either  "multiple  choice"  or  the  "wrong  conclusion",  or 
8:  The  rule  under  discussion  is  a  tabular  rule 
Then:  Include  "wrong  premise  hypothesis"  in  the  list  of  the  valid 
premise  formats  for  a  question 

Comment:  Use  wrong  hypothesis  when  there  are  non-yes/no  parms  to 

smash,  the  rule  is  unique,  and  the  action  is  either  a  table 
or  not  presented  In  multiple  choice  form.  (Uniqueness  not 
currently  checked.) 


T-RULE24 . 1 


If:  True 

Then:  To  determine  the  format  for  the  premise  part  of  the  question, 

compute  the  following:  a  random  selection  from  among  the  valid 
premise  formats  for  a  question 


HYPVALRESPONSE.RULEPK025 


Purpose:  Respond  to  the  hypothesis  that  the  goal  currently  being  discussed 
Is  this  value 

Stepl:  Apply  the  first  rule  that  Is  appropriate: 

T-RULE25.01 

If:  1)  This  hypothesized  value  Is  wrong. 
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2 )  The  number  of  domain  rules  having  values  that  were  not 
mentioned  by  the  student  is  equal  to  1,  and 

3)  It  is  not  true  that 

A:  Domain  rules  for  which  the  tutor  has  weak  evidence  that  the 
student  considered  them  to  make  his  hypothesis, 

B:  Domain  rules  that  the  tutor  believes  the  student  did  not 
consider  when  making  his  hypothesis,  or 
C:  Domain  rules  that  support  a  wrong  hypothesis 
Then:  Substep  1.  If:  There  are  domain  rules  that  the  tutor  strongly 
believes  were  used  by  the  student  to  make  his 
hypothesis 

Then:  Tell  the  student  that  domain  rules  that  the 
tutor  strongly  believes  were  used  by  the 
student  to  make  his  hypothesis  confirm  his 
hypothesis 

Substep  ii.  Generate  a  question  about  relevant  rules,  using  "key 
factors"  format  in  the  premise  part  and  "fill-in" 
format  In  the  conclusion 


T-RULE25.02 


If:  This  hypothesized  value  is  wrong 

Then:  Present  the  conclusion  that  the  goal  currently  being  discussed 
Is  (not)  this  value  [Rulepk026] 

Comment:  NB:  There  can't  be  just  one  wrongrule  and  no  missingrules;  if 
there  is  a  wrong  rule,  then  there  must  be  contrary  evidence 
the  student  did  not  consider,  by  definition.  Here  we  quiz  S 
to  see  If  he  knows  of  evidence  that  is  contrary  to  his 
hypothesis. 


T-RULE25 .03 


If:  1)  There  are  domain  rules  that  the  tutor  strongly  believes  were 
used  by  the  student  to  make  his  hypothesis, 

2)  There  are  not  domain  rules  for  which  the  tutor  has  weak 
evidence  that  the  student  considered  them  to  make  his  hypothesis, 
and 

3)  There  are  not  domain  rules  that  the  tutor  believes  thr  ,  t 

did  not  consider  when  making  his  hypothesis 

Then:  Tell  the  student  that  domain  rules  that  the  tutor  strongly 

believes  were  used  by  the  student  to  make  his  hypothesis  confirm 
his  hypothesis 

Comment:  Don’t  check  for  wrong  or  missingrules  here;  that  will  be 
taken  care  of  when  discussing  wrong  or  missing  values. 


T-RULE25 .04 


If:  1)  It  is  not  true  that 

A:  There  are  domain  rules  having  values  that  were  not  mentioned 
by  the  student, 

B:  There  are  domain  rules  that  support  a  wrong  hypothesis,  or 
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C:  There  are  domain  rules  that  the  tutor  believes  the  student 
did  not  consider  when  making  his  hypothesis, 

2)  The  number  of  domain  rules  for  which  the  tutor  has  weak 
evidence  that  the  student  considered  them  to  make  his  hypothesis 
is  equal  to  1,  and 

3)  The  number  of  values  hypothesized  by  the  student  is  greater 
than  1 

Then:  Substep  1.  If:  There  are  domain  rules  that  the  tutor  strongly 
believes  were  used  by  the  student  to  make  his 
hypothesis 

Then:  Tell  the  student  that  domain  rules  that  the 
tutor  strongly  believes  were  used  by  the 
student  to  make  his  hypothesis  confirm  his 
hypothesis 

Substep  1i.  Generate  a  question  about  the  goal  currently  being 

discussed,  using  "key  factors"  format  in  the  premise 
part  and  "fill-in"  format  in  the  conclusion 


T-RUIE25 .05 


If :  True 

Then:  Generate  a  question  about  the  goal  currently  being  discussed, 

using  "fill-in  of  known  factors"  format  in  the  premise  part  and 
the  "correct  conclusion*  format  in  the  conclusion 


IKNOWOPTION.RULEPK844 


Purpose:  Respond  to  the  student's  claim  that  he  knows  the  goal  currently 
being  discussed 

Stepl:  Apply  the  first  rule  that  Is  appropriate; 

T-RULE44 .01 


If:  Discussion  of  the  factor  Is  complete 
Then:  Say:  already-discussed 


T-RULE44.02 


If:  It  is  appropriate  now  to  request  the  reported  value  for  the  factor 
Then:  Substep  1.  Say:  sneverasked 

Substep  11.  Supply  requested  data  to  the  student,  with 

Introduction  and  related  information,  as  appropriate 
[ProceiZ] 

->  The  request  for  data  has  already  been  determined 
to  be  appropriate 


T-RULE44 .03 


If:  The  expert  has  not  reached  a  final  conclusion  about  the  factor 
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Then:  Substep  i.  Discuss  the  student's  hypothesis  [Proc814] 

Substep  iii.  Say  that  the  dialogue  is  returning  to  discussion  of 
the  goal  currently  being  discussed 
Substep  iv.  If:  Discussion  of  the  factor  is  not  complete 
Then:  Say:  mention-option 


T-RULE44 .04 


If :  True 

Then:  Substep  1.  Say:  agree-possibility 

Substep  11.  Discuss  final  conclusion  for  the  factor  by  giving 
the  final  answer  or  discussing  an  hypothesis 
[Proc005] 

->  A  prompt  should  not  be  given  before  starting 
discussion  of  this  goal 

Substep  111.  Say  that  the  dialogue  is  returning  to  discussion  of 
the  goal  currently  being  discussed 


NONEEDTOASK .RULEPK032 


Purpose:  Inform  the  student  that  it  is  not  necessary  to  request  the  reported 
value  for  the  factor  In  this  case. 

Stepl:  Apply  the  first  rule  that  is  appropriate: 

T-RULE32 .01 


If:  The  student  has  requested  the  reported  value  already 
Then:  Substep  1.  Say:  already-told 

Substep  11.  Give  the  student  the  reported  value  of  the  factor 


T-RULE32.02 


If:  This  factor  s  not  one  that  a  consultant  normally  asks 
Then:  Say:  not-askable 


T-RULE32 .03 


If:  It  Is  not  true  that  the  expert  did  find  It  necessary  to  inquire 
about  the  factor 

Then:  Explain  why  the  expert  did  not  ask  about  the  factor  [Rulepk0333 


T-RULE32 .04 


If:  1)  A  screening  question  Is  asked  before  considering  the  factor, 
and 

2)  It  Is  appropriate  now  to  request  the  reported  value  for  the 
screening  parameter 

Then:  Substep  1.  Define  the  factor  the  student  has  Inquired  about  to 
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be  the  screening  parameter 

Substep  11.  Supply  requested  data  to  the  student,  with 

Introduction  and  related  information,  as  appropriate 
[ Proc012] 

->  The  request  for  data  has  already  been  determined 
to  be  appropriate 

->  The  student  has  requested  this  data  explicitly 


T-RULE32.05 


If:  True 

Then:  Say:  premature-question 


PRESENTVALS . RULEPK026 


Purpose:  Present  the  conclusion  that  the  goal  currently  being  discussed  is 
(not)  this  value 

Stepl:  Apply  the  first  rule  that  is  appropriate: 

T-RULE26.0I 


If:  The  number  of  rules  that  conclude  this  value  is  not  equal  to  1 
Then:  Generate  a  question  about  the  goal  currently  being  discussed, 

using  “fill-in  of  known  factors*  format  in  the  premise  part  and 
the  "correct  conclusion"  format  in  the  conclusion 


T-RULE26 .02 


If:  1)  These  rules  Is  not  on  the  lesson  plan,  or 
2)  The  student  model  is  strong  for  the  rule 
Then:  Affirm  the  conclusions  made  by  the  rule  by  simply  stating  the 
key  factors  and  values  to  be  concluded 


T-RULE26 .03 


If:  The  goal  currently  being  discussed  Is  a  true/false  parameter 
Then:  Generate  a  question  about  the  goal  currently  being  discussed, 
using  the  "facts  of  the  rule’s  premise*  format  in  the  premise 
part  and  the  "correct  conclusion"  format  in  the  conclusion 


T-RULE26 .04 


If :  True 

Then:  Generate  a  question  about  the  goal  currently  being  discussed, 
using  "key  factors"  format  In  the  premise  part  and  "fill-in" 
format  In  the  conclusion 
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RULE  INTEREST . RULEPK02  7 


Purpose:  Determine  how  'Interesting*  It  Is  to  present  the  rule  at  this  time 
Stepl :  Apply  all  rules  that  are  appropriate: 

T-RUIE27.01 


If:  1)  The  rule  succeeded  In  this  context,  and 

2)  The  following  value  has  been  determined:  a  hint  value  can  be 
suggested  for  this  rule,  based  on  Its  effect  on  the  current 
hypothesis 

Then:  Modify  the  cumulative  belief  that  this  rule  should  be  presented 
by  the  sum  over  the  computed  value  as  follows: 

If  the  evidence  contributed  Is  still  Insignificant  then  .1 

If  a  new  Insignificant  value  Is  contributed  then  .1 

If  "confirmation  of  related  evidence"  then  .3 

If  a  new  significant  value  is  contributed  then  .5 

If  a  new  strongly  significant  value  is  contributed  then  .75 

If  an  Insignificant  value  becomes  significant  then  .6 

If  an  old  value  is  now  Insignificant  then  .85 

If  belief  In  an  old  value  is  strongly  contradicted  then  .95 


T-RULE27 .02 


If:  1)  The  cumulative  belief  that  this  rule  should  be  presented  is 
greater  than  .7,  and 

2)  The  student  model  Is  strong  for  the  rule  under  discussion 
Then:  Modify  the  cumulative  belief  that  this  rule  should  be  presented 
by  .3 


T-RULE27.03 


If:  The  rule  under  discussion  Is  not  on  the  lesson  plan 
Then:  Modify  the  cumulative  belief  that  this  rule  should  be  presented 
by  -.5 


T-RULE27 .04 


If:  The  following  value  has  been  determined:  how  close  the  rule  under 
discussion  Is  to  firing 

Then:  Modify  the  cumulative  belief  that  this  rule  should  be  presented 
by  the  computed  value  as  follows: 

If  the  rule  has  fired  then  .5 

If  only  case  data  remains  to  be  requested  for  the  rule  to  fire 
then  .3 

If  one  or  more  subgoals  remain  to  be  determined,  but  some 
preconditions  can  be  evaluated  then  .2 
If  otherwise  then  0.8 
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T-RULE27.05 


If:  The  rule  under  discussion  mentions  tables  or  lists  in  its  premise 
Then:  Modify  the  cumulative  belief  that  this  rule  should  be  presented 
by  .5 

Step2:  Exit  this  procedure,  returning  as  a  result:  the  cumulative  belief  that 
this  rule  should  be  presented 

RULEUSED . RULEPK028 


Purpose:  Apply  modelling  t-rules  to  determine  the  belief  that  the  rule  was 
considered  by  the  student 

Stepl:  Apply  the  first  rule  that  is  appropriate: 

T-RULE28.01 


If:  1)  The  hypothesis  does  include  values  that  can  only  be  concluded 
by  this  d-rule, 

2)  The  hypothesis  does  not  include  values  that  can  be  concluded  by 
this  d-rule,  as  well  as  other  rules,  and 

3)  There  are  values  concluded  by  the  d-rule  that  are  missing  from 
the  hypothesis 

Then:  Define  the  belief  that  this  rule  was  considered  to  be  .8 


T-RULE28.02 


If:  1)  The  hypothesis  does  include  values  that  can  only  be  concluded 
by  this  d-rule, 

2)  The  hypothesis  does  not  Include  values  that  can  be  concluded  by 
this  d-rule,  as  well  as  other  rules,  and 

3)  It  is  not  true  that  there  are  values  concluded  by  the  d-rule 
that  are  missing  from  the  hypothesis 

Then:  Define  the  belief  that  this  rule  was  considered  to  be  1.0 


T-RULE28 .03 


If:  1)  The  hypothesis  does  Include  values  that  can  only  be  concluded 
by  this  d-rule, 

2)  The  hypothesis  does  Include  values  that  can  be  concluded  by 
this  d-rule,  as  well  as  other  rules,  and 

3)  There  are  values  concluded  by  the  d-rule  that  are  missing  from 
the  hypothesis 

Then:  Define  the  belief  that  this  rule  was  considered  to  be  .6 


T-RULE28 .04 


If:  1)  The  hypothesis  does 
by  this  d-rule, 

2)  The  hypothesis  does 


Include  values  that  can  only  be  concluded 
Include  values  that  can  be  concluded  by 
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this  d-rule,  as  well  as  other  rules,  and 

3)  It  Is  not  true  that  there  are  values  concluded  by  the  d-rule 
that  are  missing  from  the  hypothesis 
Then:  Define  the  belief  that  this  rule  was  considered  to  be  .7 


T-RULE28.05 


If:  1)  The  hypothesis  does  Include  values  that  can  be  concluded  by 
this  d-rule,  as  well  as  other  rules, 

2)  The  hypothesis  does  not  Include  values  that  can  only  be 
concluded  by  this  d-rule,  and 

3)  There  are  values  concluded  by  the  d-rule  that  are  missing  from 
the  hypothesis 

Then:  Define  the  belief  that  this  rule  was  considered  to  be  -.7 


T-RUIE28.06 


If:  1)  The  hypothesis  does  Include  values  that  can  be  concluded  by 
this  d-rule,  as  well  as  other  rules, 

2)  The  hypothesis  does  not  Include  values  that  can  only  be 
concluded  by  this  d-rule,  and 

3)  It  Is  not  true  that  there  are  values  concluded  by  the  d-rule 
that  are  missing  from  the  hypothesis 

Then:  Define  the  belief  that  this  rule  was  considered  to  be  .4 


T-RULE28 .07 


If:  1)  The  hypothesis  does  not  Include  values  that  can  only  be 
concluded  by  this  d-rule, 

2)  The  hypothesis  does  not  include  values  that  can  be  concluded  by 
this  d-rule,  as  well  as  other  rules,  and 

3)  There  are  values  concluded  by  the  d-rule  that  are  missing  from 
the  hypothesis 

Then:  Define  the  belief  that  this  rule  was  considered  to  be  -1.0 


T-RULE28.08 


If:  1)  The  hypothesis  does  not  include  values  that  can  only  be 
concluded  by  this  d-rule, 

2)  The  hypothesis  does  not  Include  values  that  can  be  concluded  by 
this  d-rule,  as  well  as  other  rules,  and 

3)  It  Is  not  true  that  there  are  values  concluded  by  the  d-rule 
that  are  missing  from  the  hypothesis 

Then:  Define  the  belief  that  this  rule  was  considered  to  be  0 

Step2:  Record  that  the  tutor’s  belief  in  the  student’s  knowledge  of  the  rule 
Is  the  belief  that  this  rule  was  considered 
Step3:  Exit  this  procedure,  returning  as  a  result:  the  belief  that  this  rule 
was  considered 
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RULEUSEDSREF . RULEPK029 


Purpose:  In  view  of  the  model  for  all  other  rules,  apply  self-referencing 

t-rules  to  determine  whether  the  rule  was  considered  by  the  student 

Stepl :  If:  The  belief  that  this  rule  was  considered  is  equal  to  1000 
Then:  Exit  this  procedure 
Step2:  Apply  all  rules  that  are  appropriate: 

T-RULE29 .01 


If:  You  believe  that  this  rule  was  considered,  it  concludes  a  value 
present  in  the  student's  hypothesis,  and  no  other  rule  that 
mentions  this  value  is  believed  to  have  been  considered 
Then:  Modify  the  cumulative  belief  that  this  rule  was  considered  by 
.4 


T-RULE29.02 


If:  1)  The  belief  that  the  student  knows  this  rule  is  not  zero,  and 
2)  The  belief  that  this  rule  was  considered  and  the  belief  that 
the  student  knows  this  rule  are  of  the  same  sign 
Then:  Modify  the  cumulative  belief  that  this  rule  was  considered  by 
the  belief  that  the  student  knows  this  rule 
Else:  Record  on  the  -genda  that  there  is  an  inconsistency  in  the 
knowledge  model  for  this  rule 


T-RULE29.03 


If:  1)  The  rule  has  been  discussed,  and 

2)  The  belief  that  this  rule  was  considered  Is  not  negative 
Then:  Modify  the  cumulative  belief  that  this  rule  was  considered  by 

.7 


T-RULE29.84 


If:  1)  The  belief  that  this  rule  was  considered  is  less  than  the 
threshold  for  strong  evidence,  and 

2)  You  have  examined  the  the  factors  associated  with  this  rule, 
selecting  every  factor  for  which  Discussion  of  the  factor  is  not 
complete 

Then:  Substep  1.  Modify  the  cumulative  belief  that  this  rule  was 
considered  by  -.6 

Substep  11.  Record  on  the  agenda  that  has  subgoals  that  need  to 
be  discussed 


T-RULE29.05 


If:  This  domain  rule  contains  a  factor  that  appears  In  several  rules, 
none  of  which  are  believed  to  have  been  considered  to  make  the 
hypothesis 


t 
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Then:  Substep  i.  Modify  the  cumulative  belief  that  this  rule  was 
considered  by  -.3 

Substep  1i.  Record  on  the  agenda  that  this  domain  rule  contains 
a  factor  that  appears  In  several  rules,  none  of 
which  are  believed  to  have  been  considered  to  make 
the  hypothesis 

Step3:  Record  that  the  tutor's  belief  in  the  student's  knowledge  of  the  rule 
is  the  cumulative  belief  that  this  rule  was  considered 

Step4:  Exit  this  procedure,  returning  as  a  result:  the  cumulative  belief  that 
this  rule  was  considered 

SAPPLIEORULE .RULEPK041 


Purpose:  Determine  whether  the  student  knows  enough  to  use  the  rule 
Stepl:  Apply  all  rules  that  are  appropriate: 

T-RULE41 .01 


If:  The  record  from  past  interactions  with  the  student  that  he  knows 
this  rule  is  zero 

Then:  Modify  the  cumulative  belief  that  the  student  applied  this  rule 
by  Determine  the  a  priori  belief  that  the  student  knows  the 
rule  [Ru1epk040] 

Else:  Modify  the  cumulative  belief  that  the  student  applied  this  rule 
by  the  record  from  past  interactions  with  the  student  that  he 
knows  this  rule 


T-RULE41 .82 


If:  Mentlon-thls-sesslon  of  the  rule 

Then:  Modify  the  cumulative  belief  that  the  student  applied  this  rule 
by  the  threshold  for  strong  evidence 


T-RULE41 .03 


If:  The  rule  has  been  discussed  before 

Then:  Modify  the  cumulative  belief  that  the  student  applied  this  rule 
by  .2 


T-RULE41 .04 


If:  1)  The  rule  has  not  been  discussed,  and 

2)  It  Is  not  true  that  the  student  knows  enough  about  relevant 
subgoals  so  that  he  can  evaluate  the  premise  of  the  rule 
Then:  Modify  the  cumulative  belief  that  the  student  applied  this  rule 
by  -1.0 

Comment:  This  sets  the  SCANTAPPir  property  of  the  rule  If  one  or  more 
clauses  can't  be  applied  by  the  student  due  to  missing 


t 
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subgoals  In  his  knowledge  nodal 

Step2:  Record  that  the  belief  that  the  student  applied  this  rule  of  the  rule 
Is  the  cumulative  belief  that  the  student  applied  this  rule 
Step3:  Exit  this  procedure,  returning  as  a  result:  the  cumulative  belief  that 
the  student  applied  this  rule 

SUGGESTHYP . RULEPK030 


Purpose:  Suggest  a  rule  to  the  student  In  the  form  of  help  for  reaching  an 
hypothesis 

Stepl:  Apply  the  first  rule  that  is  appropriate: 

T-RULE30.01 


If:  1)  The  premise  of  the  rule  being  considered  Includes  an  Iteration, 
or 

2)  A:  The  student  model  Is  negative  for  the  rule  being  considered, 
and 

B:  The  rule  being  considered  Is  not  on  the  lesson  plan 
Then:  Affirm  the  conclusions  made  by  the  rule  being  considered  by 
simply  stating  the  key  factors  and  values  to  be  concluded 


T-RULE3B .02 


If:  The  hypothesis  topic  Is  a  true/false  parameter 

Then:  Generate  a  question  about  the  hypothesis  topic,  using  the  "facts 
of  the  rule’s  premise*  format  In  the  premise  part  and  the 
"correct  conclusion"  format  In  the  conclusion 


T-RULE30 .03 


If:  1)  The  rule  being  considered  Is  not  the  only  rule  that  concludes 
this  value  for  the  hypothesis  topic,  or 
2)  The  factor  being  focussed  upon 
Then:  Generate  a  question  about  the  hypothesis  topic,  using  "key 

factors"  format  In  the  premise  part  and  "fill-in"  format  in  the 
conclusion 


T-RULE30 .04 


If:  True 

Then:  Generate  a  question  about  the  hypothesis  topic,  using  "fill-in 
of  known  factors*  format  In  the  premise  part  and  the  "correct 
conclusion"  format  In  the  conclusion 
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SUGGESTRULE .RULEPK031 


Purpose:  Present  the  rule  being  considered 
Stepl:  Apply  the  first  rule  that  Is  appropriate: 
T-RULE31 .01 


If:  1)  The  rule  has  fired,  and 

2)  The  premise  of  the  rule  being  considered  Includes  an  Iteration 
Then:  Generate  a  question  about  the  goal  currently  being  discussed, 
using  the  "facts  of  the  rule's  premise*  format  In  the  premise 
part  and  the  "correct  conclusion*  format  In  the  conclusion 


T-RULE31 .02 


If:  The  rule  has  fired 

Then:  Substep  1.  Select  a  question  type  suitable  for  the  rule  being 
considered  [Rulepk024] 

Substep  11.  Generate  a  question  based  upon  the  rule  being 
considered  and  evaluate  the  student’s  response 
Substep  111.  Record  that  the  rule  being  considered  has  been 
discussed 


T-RULE31 .03 


If:  1)  What  has  been  mentioned  In  the  recent  context  of  the 

Interaction  Is  either  a  deeper  subgoal  or  a  subtopic  relevant  to 
the  current  goal,  and 

2)  Discussion  of  the  factor  being  focussed  upon  Is  not  complete 
Then:  Substep  1.  Say:  sglntro 

Substep  11.  Discuss  the  goal  with  the  student  In  a  goal-directed 
mode  [Proc001] 

Substep  111.  Say:  focusrulelntro 

Substep  1v.  Discuss  the  rule  being  considered  clause-by-clause 
[Proc004] 


T-RULE31 .04 


If:  1)  The  number  of  factors  appearing  In  this  rule  which  need  to  be 
asked  by  the  student  Is  zero,  and 

2)  The  number  of  subgoals  remaining  to  be  determined  before  this 
rule  can  be  applied  Is  equal  to  1 
Then:  Substep  1.  Say:  suggestsg 

Substep  11.  Discuss  the  goal  with  the  student  In  a  goal-directed 
mode  [Proc001] 

•>  A  prompt  should  not  be  given  before  starting 
discussion  of  this  goal 

Substep  111.  Wrap  up  the  discussion  of  the  rule  being  considered 
[Proc017] 
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T-RULE31 .05 


If:  1)  The  premise  of  the  rule  being  considered  includes  an  iteration, 

2)  The  number  of  subgoals  remaining  to  be  determined  before  this 
rule  can  be  applied  is  greater  than  1, 

3)  A:  What  has  been  mentioned  in  the  recent  context  of  the 
interaction  is  a  hint  about  a  rule, 

6:  The  rule  hinted  about  is  the  rule  being  considered,  and 
C:  The  number  of  subgoals  remaining  to  be  determined  before 
this  rule  can  be  applied  is  not  zero,  or 

4)  The  student  model  Is  negative  for  the  rule  being  considered 
Then:  Substep  1.  Select  an  introduction  for  beginning  discussion  of 

the  rule  being  considered  [Rulepk038] 

Substep  11.  Discuss  the  rule  being  considered  clause-by-clause 
[Proc004] 


T-RULE31 .06 


If:  1)  A  hint  value  can  be  suggested  for  this  rule,  based  on  its 
effect  on  the  current  hypothesis,  and 

2)  A:  What  has  been  mentioned  in  the  recent  context  of  the 
.  interaction  is  not  a  hint  about  a  rule,  or 
B:  The  rule  hinted  about  is  not  the  rule  being  considered 
Then:  Substep  1.  Say:  valu-hint 

Substep  11.  Define  the  rule  hinted  about  to  be  the  rule  being 
considered 

Substep  111.  Define  what  has  been  mentioned  In  the  recent 

context  of  the  Interaction  to  be  a  hint  about  a 
rule 


T-RULE31.B7 


If:  1)  The  number  of  subgoals  remaining  to  be  determined  before  this 
rule  can  be  applied  is  zero, 

2)  A:  1)  What  has  been  mentioned  In  the  recent  context  of  the 

Interaction  is  a  hint  about  a  rule,  and 

2)  The  rule  hinted  about  Is  the  rule  being  considered, 

B:  The  rule  being  considered  Is  on  the  lesson  plan,  or 

C:  It  Is  not  true  that  the  student  model  Is  strong  for  the  rule 

being  considered, 

3)  This  topic  Is  not  a  true/false  parameter,  and 

4)  The  rule  being  considered  Is  the  only  rule  that  concludes  this 
value  for  the  goal  currently  being  discussed 

Then:  Generate  a  question  about  the  goal  currently  being  discussed, 

using  "given  new  facts"  format  In  the  premise  part  and  "fill-in" 
format  In  the  conclusion 


T-RUIE31.88 


If:  1)  The  number  of  subgoals  remaining  to  be  determined  before  this 
rule  can  be  applied  Is  zero,  and 

2)  A:  1)  What  has  been  mentioned  In  the  recent  context  of  the 
Interaction  Is  a  hint  about  a  rule,  and 
2)  The  rule  hinted  about  Is  the  rule  being  considered, 

B:  The  rule  being  considered  Is  on  the  lesson  plan,  or 


S 


> 
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C:  It  is  not  true  that  the  student  model  is  strong  for  the  rule 
being  considered 

Then:  Generate  a  question  about  the  goal  currently  being  discussed, 

using  "fill-in  undiscussed  facts"  format  in  the  premise  part  and 
the  "correct  conclusion*  format  in  the  conclusion 


T-RULE31 .99 


If:  True 

Then:  Substep  1.  ***missing*»*  suggestinfoclasses 
Substep  1i.  Say:  hintconcls 

Substep  ill.  Oefine  what  has  been  mentioned  in  the  recent 

context  of  the  interaction  to  be  a  hint  about  a 
rule 

Substep  iv.  Define  the  rule  hinted  about  to  be  the  rule  being 
considered 


♦ 
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